-- Logs begin at Wed 2022-06-15 15:00:54 UTC, end at Wed 2022-06-15 16:16:17 UTC. -- Jun 15 15:35:37.369867 dv-cinder-845990-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 15 15:35:37.377156 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 15 15:35:37 2022] *** Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: nodename: dv-cinder-845990-1-iscsi Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: machine: x86_64 Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: clock source: unix Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: pcre jit disabled Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: detected number of CPU cores: 4 Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: current working directory: / Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: detected binary path: /usr/bin/uwsgi-core Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: your processes number limit is 47680 Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: your memory page size is 4096 bytes Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: detected max file descriptor number: 2048 Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: lock engine: pthread robust mutexes Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: thunder lock: enabled Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 15 15:35:37.380994 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 15 15:35:37.425048 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: Python main interpreter initialized at 0x55d59deb09c0 Jun 15 15:35:37.425048 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: python threads support enabled Jun 15 15:35:37.425048 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: your server socket listen backlog is limited to 100 connections Jun 15 15:35:37.425048 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: your mercy for graceful operations on workers is 90 seconds Jun 15 15:35:37.425413 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: mapped 403077 bytes (393 KB) for 2 cores Jun 15 15:35:37.425482 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: *** Operational MODE: preforking *** Jun 15 15:35:37.425737 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: *** uWSGI is running in multiple interpreter mode *** Jun 15 15:35:37.425737 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: spawned uWSGI master process (pid: 86629) Jun 15 15:35:37.425938 dv-cinder-845990-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 15 15:35:37.427860 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: spawned uWSGI worker 1 (pid: 86639, cores: 1) Jun 15 15:35:37.427860 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: spawned uWSGI worker 2 (pid: 86640, cores: 1) Jun 15 15:35:37.427860 dv-cinder-845990-1-iscsi devstack@keystone.service[86629]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 15 15:35:42.944468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 15:35:42.944468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: from cryptography.utils import int_from_bytes Jun 15 15:35:42.945219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 15:35:42.945219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: from cryptography.utils import int_from_bytes Jun 15 15:35:42.945333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 15:35:42.945333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: from cryptography.utils import int_from_bytes Jun 15 15:35:42.945732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 15:35:42.945732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: from cryptography.utils import int_from_bytes Jun 15 15:35:42.994022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 15:35:42.994215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 15:35:42.994412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] command line args: [] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 15:35:42.994502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 15:35:42.994626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 15:35:42.994709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 15:35:42.994845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] command line args: [] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 15:35:42.994932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ================================================================================ {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 15:35:42.995120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 15:35:42.995205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] admin_token = **** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.995302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ================================================================================ {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 15:35:42.995386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] config_dir = [] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.995548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.995632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] admin_token = **** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.995738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] config_source = [] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.995816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] config_dir = [] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.995922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] debug = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.996012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.996122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.997412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] config_source = [] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.997412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] debug = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.997533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.997602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.997694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.997756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.997756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.997864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.997934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.998000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.998000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] list_limit = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.998114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.998208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] log_config_append = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.998796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] list_limit = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.998880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.998944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] log_config_append = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] log_dir = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] log_file = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] log_dir = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] log_file = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:42.999895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.000198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.000277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.000417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.000506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.000581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.000705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.000789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.000868 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.000991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.001073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.001146 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.001256 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.001342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.001517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.001618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.001756 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.001846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.002017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.002113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.002296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.002514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.002701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] publish_errors = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.002816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.002965 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.003048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] publish_errors = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.003199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.003295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.003369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.003439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.003611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.003688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.003847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.003933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.004087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.004164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] standard_threads = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.004320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.004446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] standard_threads = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.004615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.004732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.004732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.004953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.005068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] use_journal = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.005190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.005313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] use_json = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.005429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] use_journal = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.005543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] use_stderr = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.005657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] use_json = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.005777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] use_syslog = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.005903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] use_stderr = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.005903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] use_syslog = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.006119 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.006239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 15:35:43.006360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.006482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.006606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.006722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.006863 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.006988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.007105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.007218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.007342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.007458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.007572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.007686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.007799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.007918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.008041 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.008156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.008271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.008392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.008513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.008628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.008750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.008866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.008987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.009108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.009223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.009337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.009460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.009582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.009703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.009824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.009946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.010067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.010265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.010383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.010504 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.010619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.010755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.010884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.011010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.011137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.011253 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.011373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.011489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.011604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.011730 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.011847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.011967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.012082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.012204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.012326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.012450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.012573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.012689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.012803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.012917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.013042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.013164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.013285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.013400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.013520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.013637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.013759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.013875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.013988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.014107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.014223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.014337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.014466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.014583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.014697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.014829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.014944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.015058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.015185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.015302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.015422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.015543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.015659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.015774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.015963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.016080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.016194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.016315 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.016429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.016544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.016666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.016784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.016904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.017020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.017140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.017262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.017386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.017502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.017617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.017731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.017836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.017944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.018050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.018148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.018244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.018340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.018443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.018541 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.018654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.018810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.018887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.019006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.019122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.019271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.019404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.019522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.019637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.019758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.019875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.019997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.020121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.020238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.020353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.020474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.020591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.020714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.020823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.020946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.021067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.021191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.021317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.021443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.021560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.021667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.021789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.021906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.022025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.022140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.022261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.022378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.022504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.022623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.022754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.022865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.022990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.023116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.023234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.023471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.023588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.023703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.023826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.023943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.024059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.024175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.024291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.024405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.024532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.024896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.025012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.025127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.025243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.025358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.025482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.025599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.025705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.025819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.025934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.026050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.026173 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.026291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.026406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.026522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.026637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.026766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.026897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.027016 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.027131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.027256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.027373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.027489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.027680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.027799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.027923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.028039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.028161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.028279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.028406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.028532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.028668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.028787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.028904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.029019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.029135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.029251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.029377 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.029494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.029623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.029779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.029909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.030027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.030152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.030270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.030388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.030504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.030619 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.030777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.030913 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.031062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.031183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.031347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.031469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.031593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.031711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] auth.external = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.031848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] auth.external = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.031969 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.032084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.032200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.032383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.032512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.032640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] auth.password = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.032758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.032875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] auth.token = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.032991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] auth.password = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.033107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] auth.token = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.033230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.033357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.033474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.033592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.033714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.033832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.033947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.034072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.034189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.034306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.034422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.034539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.034656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.034803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] credential.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.034925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.035042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.035159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] credential.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.035275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.035391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.035516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.035642 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.035760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.035877 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.035993 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.036110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.036234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.036352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.036352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.036571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.036695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.036813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.036937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.037063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.037063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.037281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.037398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.037514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.037639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.037765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.037891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.038009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.038126 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.038242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.038367 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.038486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.038603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] federation.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.038718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.038881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.038998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] federation.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.039127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.039246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.039362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.039479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.039479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.039697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.039822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.039939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.039939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.040173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.040173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.040392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.040517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.040634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.040634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.040853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.040976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.041103 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.041230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.041353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.041471 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.041587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.041703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.041819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.042012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.042132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.042255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.042373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.042490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.042614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.042766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.042886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.043004 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.043121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.043237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.043366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.043486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.043603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.043719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.043836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.043952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.044079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.044197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.044314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.044431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.044548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.044670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.044816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.044930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.045538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.045665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.045784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.045900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.046028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.046148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.046265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.046381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.046497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.046613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.046794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.046915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.047032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.047149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.047266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.047382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.047510 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.047629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.047746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.047862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.047978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.048094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.048219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.048338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.048454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.048571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.048688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.048804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.048928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.049046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.049163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.049280 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.049396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.049513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.049637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.049755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.049871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.049986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.050103 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.050218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.050344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.050463 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.050579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.050695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.050829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.050947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.051084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.051210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.051328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.051444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.051561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.051677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.051802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.051921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.052037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.052153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.052270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.052386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.052512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.052630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.052745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.052862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.052978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.053094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.053219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.053337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.053454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.053570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.053693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.053812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.053936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.054054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.054171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.054286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.054403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.054528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.054646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.054806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.054925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.055042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.055158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.055287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.055406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.055523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.055640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.055757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.055874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.055998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.056116 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.056233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.056349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.056466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.056583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.056708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.056860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.056981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.057097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.057214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.057330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.057455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.057573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.057690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.057807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.057923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.058039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.058164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.058282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.058399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.058515 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.058631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.058766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.058900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.059019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.059135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.059251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.059368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.059484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.059608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.059727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.059845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.059961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.060077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.060193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.060318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.060436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.060552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.060702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.061348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.061470 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.061600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.061719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.061837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.061953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.062077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.062194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.062318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.062437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.062553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.062670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.062806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.062925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.063054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.063173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.063290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.063406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.063522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.063638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.063763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] resource.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.063881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.063999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.064116 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] resource.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.064232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.064348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.064473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.064591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.064708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.064825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.064941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.065058 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.065183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.065301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.065418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.065535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.065652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.065768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.065955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.066076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.066193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] role.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.066309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.066426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] role.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.066543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] role.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.066667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.066832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] role.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.066950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.067066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.067184 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.067300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.067429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.067548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.067665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.067782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.067898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.068024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.068144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.068261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.068378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.068494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.068611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.068735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.068859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.068977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.069093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.069211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.069327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.069452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.069569 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.069686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.069803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.069919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.069919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.070145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.070145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.070366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.070483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.070599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.070715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.070865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.070985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.071102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.071218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.071339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.071456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.071582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.071582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.071802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.071918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.072035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.072035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.072262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.072381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.072497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.072614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.072730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.072854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.072972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.073089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.073205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.073321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.073438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.073562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.073680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.073796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.073912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.074028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.074145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.074268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.074386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.074503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] token.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.074626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.074789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.074909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] token.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.075039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.075159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.075276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.075392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.075392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.075611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.075611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.075839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.075956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.076073 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.076189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.076329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.076455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.076575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.076692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.076808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.076925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.077041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.077166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.077284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.077400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.077517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.077635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.077751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.077875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.077994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.078110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.078226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.078343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.078459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.078459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.078688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.078822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.078940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.connection = **** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.079073 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.connection = **** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.079205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.079332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.079451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.079568 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.079695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.079813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.079931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.080048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.080171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.080289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.080415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.080534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.080652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.080770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.080888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.081005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.081131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.081249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.081370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.081487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.081605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.081722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.081848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.081967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.082085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.082202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.082319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.082436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.082561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.082680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.082851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.082969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.083087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.083204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.083333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.083333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 15:35:43.083557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 15:35:43.083557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 15:35:43.198345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.198507 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.198662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.198764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.198890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.198973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.207770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.207951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.235860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.235971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.236094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.236177 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.291770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.291913 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.341065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.341298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.342258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.342499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.409345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.409571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.409796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.410018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.410240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.410460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.413349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.413589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.413819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.414044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.414269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.414491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.435707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.435707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.435987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436174 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.436776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.447415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.447620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.461069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.461275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.461432 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.461608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.462089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.462089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.462334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.462467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86640) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.465845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.466421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.466658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.466957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.467680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.467914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 15:35:43.480174 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.480390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 15:35:43.480730 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.480893 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.480987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 15:35:43.481107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 15:35:43.481370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.481452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.481629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 15:35:43.481709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.481887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.482074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.482155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.482341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.482629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.482923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.493110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.493387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.493658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.493817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.493936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.494097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.494365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.494643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.502139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.502974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.505609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.505890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.506174 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.506387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.506469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.506671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.506770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.506980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.507063 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.507253 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.507344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.507530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.507613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.507790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.507868 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.508059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.508147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.508327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.508407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.508594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.508865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.509131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.536495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.537212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.538116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.538395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.538867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.539149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.543794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.544083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.544367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.544447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.544732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.545038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.551929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.552209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.552669 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.552951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.556720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.557010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.557295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.557457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.557578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.557746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.557854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.558030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.558129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.558314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.558403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.558589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.558670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.558892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.559167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.559444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.577921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.578197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.578849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.579130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.582767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.583722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.586396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.586692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.587407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.587701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.591833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.593402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.595964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.597172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.599772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.600071 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.600360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.600647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.600864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.600945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.601165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.601249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.601452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.601738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.602018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.602303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.620376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.620658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.620924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.621186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.621561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.621849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.622115 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.622379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.629180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.630398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.630949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.631233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.632205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.632482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.634298 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.634582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.635586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.635878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.636180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.636490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.636781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.637445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.637746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.638034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.650926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.651205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.651488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.651769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.652066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.652355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.652438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.652638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.652719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.652921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.653204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.653709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.653993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.654272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.671401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.673112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.674674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.676425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.678017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.679652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.679772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.681398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.681486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.681707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.682006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.683202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.683487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.683772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.689403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.691059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.691210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.692834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.694305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.694596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.694903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.695181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.696106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.696395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.696677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.696952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.705212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.705455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.705735 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.708379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.708670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.708952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.713671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86640) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.713951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.714230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.714506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.714803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.715099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.715375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.715644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.715925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.716300 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.716575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.716889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.717482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 15:35:43.717652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.717734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.717933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86640) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.718023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.718232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.718522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.718818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.719093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.719367 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.719643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.720013 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.720288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.720564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.720834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.721097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 15:35:43.755859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d59deb09c0 pid: 86640 (default app) Jun 15 15:35:43.759135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d59deb09c0 pid: 86639 (default app) Jun 15 15:35:43.759253 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b61fe-7d5b-4f87-802d-d06c69fba1a7 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:43.759557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b61fe-7d5b-4f87-802d-d06c69fba1a7 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:43.759842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b61fe-7d5b-4f87-802d-d06c69fba1a7 None None] PATH_INFO: `/v3` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:43.764581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Wed Jun 15 15:35:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:35:51.487023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abbcd5-c780-4183-b633-af677188e03b None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:51.487513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abbcd5-c780-4183-b633-af677188e03b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:51.487797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abbcd5-c780-4183-b633-af677188e03b None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:51.492366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1/2] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:35:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:35:51.498863 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:51.499305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:51.499423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:51.568509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: INFO dbcounter [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] Registered counter for database keystone Jun 15 15:35:51.589785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG oslo_db.sqlalchemy.engines [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86640) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 15:35:51.595948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writer thread running {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 15:35:51.772936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG passlib.handlers.bcrypt [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86640) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 15:35:51.773271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG passlib.handlers.bcrypt [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86640) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 15:35:51.827666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:35:51.829378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:35:51.950694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 2/3] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:35:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 15 15:35:52.116265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abbcd5-c780-4183-b633-af677188e03b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:52.186499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: INFO dbcounter [None req-c2abbcd5-c780-4183-b633-af677188e03b None None] Registered counter for database keystone Jun 15 15:35:52.205304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG oslo_db.sqlalchemy.engines [None req-c2abbcd5-c780-4183-b633-af677188e03b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86639) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 15:35:52.211444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writer thread running {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 15:35:52.329501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abbcd5-c780-4183-b633-af677188e03b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:52.333748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6bc6e-4e91-44d1-90a9-f950e4800ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:52.334413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6bc6e-4e91-44d1-90a9-f950e4800ced admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:52.334697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6bc6e-4e91-44d1-90a9-f950e4800ced admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:52.335002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6bc6e-4e91-44d1-90a9-f950e4800ced admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:52.343547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6bc6e-4e91-44d1-90a9-f950e4800ced admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:52.395955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d4c6bc6e-4e91-44d1-90a9-f950e4800ced admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 15:35:52.396300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6bc6e-4e91-44d1-90a9-f950e4800ced admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:52.405936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 2/4] 10.222.0.62 () {58 vars in 1324 bytes} [Wed Jun 15 15:35:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:35:52.411823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:52.428305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2ffcb-4cca-4e99-8c76-651fad9a134c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:52.432731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37c354-cbe5-436b-a5f1-34989a99ad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:52.433951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37c354-cbe5-436b-a5f1-34989a99ad74 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:52.434242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37c354-cbe5-436b-a5f1-34989a99ad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:52.434520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37c354-cbe5-436b-a5f1-34989a99ad74 admin admin] PATH_INFO: `/v3/projects/87109ec03f124710b1afe403b1e173c0` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:52.436635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37c354-cbe5-436b-a5f1-34989a99ad74 admin admin] RBAC: Authorizing `identity:get_project(project_id=87109ec03f124710b1afe403b1e173c0)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:52.488147 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0c37c354-cbe5-436b-a5f1-34989a99ad74 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 15:35:52.488484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37c354-cbe5-436b-a5f1-34989a99ad74 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:52.493150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 3/5] 10.222.0.62 () {56 vars in 1207 bytes} [Wed Jun 15 15:35:52 2022] GET /identity/v3/projects/87109ec03f124710b1afe403b1e173c0 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:35:54.564650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d92b5-7dcc-4098-89f4-a6e419cdf353 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:54.565176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d92b5-7dcc-4098-89f4-a6e419cdf353 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:54.565271 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d92b5-7dcc-4098-89f4-a6e419cdf353 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:54.569359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 3/6] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:35:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:35:54.574570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24c319-dab1-4501-b8a0-d1d741df8024 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:54.574961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24c319-dab1-4501-b8a0-d1d741df8024 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:54.575160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24c319-dab1-4501-b8a0-d1d741df8024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:54.594029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-cc24c319-dab1-4501-b8a0-d1d741df8024 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:35:54.595583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cc24c319-dab1-4501-b8a0-d1d741df8024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:35:54.603411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 4/7] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:35:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 15:35:54.625702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d92b5-7dcc-4098-89f4-a6e419cdf353 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:54.637481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d92b5-7dcc-4098-89f4-a6e419cdf353 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:54.641651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1c006-47f6-4a77-b4f3-42c6846e6daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:54.642273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1c006-47f6-4a77-b4f3-42c6846e6daf admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:54.642556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1c006-47f6-4a77-b4f3-42c6846e6daf admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:54.642865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1c006-47f6-4a77-b4f3-42c6846e6daf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:54.650633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1c006-47f6-4a77-b4f3-42c6846e6daf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:54.652425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1c006-47f6-4a77-b4f3-42c6846e6daf admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:54.661764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 4/8] 10.222.0.62 () {58 vars in 1324 bytes} [Wed Jun 15 15:35:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:35:54.666757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24c319-dab1-4501-b8a0-d1d741df8024 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:54.676111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24c319-dab1-4501-b8a0-d1d741df8024 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:54.680436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb3296-c946-4183-aca8-a1243e648fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:54.681279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3296-c946-4183-aca8-a1243e648fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:54.681565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3296-c946-4183-aca8-a1243e648fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:54.681843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3296-c946-4183-aca8-a1243e648fa0 admin admin] PATH_INFO: `/v3/users/38729f2fb8864984b35336009a142811` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:54.683250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb3296-c946-4183-aca8-a1243e648fa0 admin admin] RBAC: Authorizing `identity:get_user(user_id=38729f2fb8864984b35336009a142811)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:54.685083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb3296-c946-4183-aca8-a1243e648fa0 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:54.686617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 5/9] 10.222.0.62 () {56 vars in 1198 bytes} [Wed Jun 15 15:35:54 2022] GET /identity/v3/users/38729f2fb8864984b35336009a142811 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:35:56.837124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb00bdf-ba47-444d-973f-b31d5228682c None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:56.837626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb00bdf-ba47-444d-973f-b31d5228682c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:56.837626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb00bdf-ba47-444d-973f-b31d5228682c None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:56.839088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 5/10] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:35:56.844808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf38965-02ce-437c-be1b-819b33065194 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:56.845336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf38965-02ce-437c-be1b-819b33065194 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:56.845424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf38965-02ce-437c-be1b-819b33065194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:56.861614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-bdf38965-02ce-437c-be1b-819b33065194 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:35:56.862807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bdf38965-02ce-437c-be1b-819b33065194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:35:56.883416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0f697-d07c-4420-9b0e-dafa9dfd17dc None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:56.883709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0f697-d07c-4420-9b0e-dafa9dfd17dc None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:56.883989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0f697-d07c-4420-9b0e-dafa9dfd17dc None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:56.885333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 6/11] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:35:56.890852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6e339-e4d4-4f9f-9f6a-e2e89b09ee4f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:56.891138 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6e339-e4d4-4f9f-9f6a-e2e89b09ee4f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:56.891414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6e339-e4d4-4f9f-9f6a-e2e89b09ee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:56.930052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG passlib.handlers.bcrypt [None req-01f6e339-e4d4-4f9f-9f6a-e2e89b09ee4f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86639) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 15:35:56.930477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG passlib.handlers.bcrypt [None req-01f6e339-e4d4-4f9f-9f6a-e2e89b09ee4f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86639) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 15:35:56.943703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 6/12] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:35:56.974226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf38965-02ce-437c-be1b-819b33065194 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:56.976411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-01f6e339-e4d4-4f9f-9f6a-e2e89b09ee4f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:35:56.978019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-01f6e339-e4d4-4f9f-9f6a-e2e89b09ee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:35:56.998101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:35:57.079553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 7/13] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:35:57.081579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf38965-02ce-437c-be1b-819b33065194 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:57.100901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6e339-e4d4-4f9f-9f6a-e2e89b09ee4f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:57.114844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550bdd7-12df-4036-9b8a-af87b503585e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:57.115644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550bdd7-12df-4036-9b8a-af87b503585e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:57.115934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550bdd7-12df-4036-9b8a-af87b503585e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:57.116217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550bdd7-12df-4036-9b8a-af87b503585e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:57.128113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550bdd7-12df-4036-9b8a-af87b503585e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-5550bdd7-12df-4036-9b8a-af87b503585e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:35:57.139354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:35:57.140274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:35:57.141128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 7/14] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:35:57.146199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550bdd7-12df-4036-9b8a-af87b503585e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:57.200855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6e339-e4d4-4f9f-9f6a-e2e89b09ee4f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:57.203158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:35:57.227132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:35:57.241094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abcbd1-9544-4cc7-8414-b93674b30c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:57.241989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abcbd1-9544-4cc7-8414-b93674b30c51 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:57.242280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abcbd1-9544-4cc7-8414-b93674b30c51 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:57.242561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abcbd1-9544-4cc7-8414-b93674b30c51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:57.246023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550bdd7-12df-4036-9b8a-af87b503585e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:57.279865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9af295-fc18-41b3-aac9-d6ce00c842e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:57.280532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9af295-fc18-41b3-aac9-d6ce00c842e9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:57.280820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9af295-fc18-41b3-aac9-d6ce00c842e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:57.281101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9af295-fc18-41b3-aac9-d6ce00c842e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:57.281959 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9af295-fc18-41b3-aac9-d6ce00c842e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:35:57.282273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9af295-fc18-41b3-aac9-d6ce00c842e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:57.283711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9af295-fc18-41b3-aac9-d6ce00c842e9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:57.295789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 8/15] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:35:57.301203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9af295-fc18-41b3-aac9-d6ce00c842e9 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:57.313749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abcbd1-9544-4cc7-8414-b93674b30c51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:57.315337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abcbd1-9544-4cc7-8414-b93674b30c51 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:57.399435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9af295-fc18-41b3-aac9-d6ce00c842e9 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:57.412894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 8/16] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 15:35:57.418322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abcbd1-9544-4cc7-8414-b93674b30c51 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:57.421527 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:35:57.434064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b67e0-7cbe-4698-b8bf-1d4ce6e007b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:57.435012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b67e0-7cbe-4698-b8bf-1d4ce6e007b9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:57.435286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b67e0-7cbe-4698-b8bf-1d4ce6e007b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:57.435564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b67e0-7cbe-4698-b8bf-1d4ce6e007b9 None admin] PATH_INFO: `/v3/users/38729f2fb8864984b35336009a142811` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:57.436788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b67e0-7cbe-4698-b8bf-1d4ce6e007b9 None admin] RBAC: Authorizing `identity:get_user(user_id=38729f2fb8864984b35336009a142811)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:57.438238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b67e0-7cbe-4698-b8bf-1d4ce6e007b9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:57.440077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 9/17] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:35:57 2022] GET /identity/v3/users/38729f2fb8864984b35336009a142811 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:35:57.445080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b67e0-7cbe-4698-b8bf-1d4ce6e007b9 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:57.459542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:35:57.524202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abcbd1-9544-4cc7-8414-b93674b30c51 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:57.556598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b67e0-7cbe-4698-b8bf-1d4ce6e007b9 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:57.561569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc656d6-0dcd-4a90-abe8-0d9930e60c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:57.562363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc656d6-0dcd-4a90-abe8-0d9930e60c28 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:57.562645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc656d6-0dcd-4a90-abe8-0d9930e60c28 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:57.562952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc656d6-0dcd-4a90-abe8-0d9930e60c28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:57.580683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc656d6-0dcd-4a90-abe8-0d9930e60c28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:57.582139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc656d6-0dcd-4a90-abe8-0d9930e60c28 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:57.589768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887581b0-8d0a-4ba9-be50-38b9aada3f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:57.590557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887581b0-8d0a-4ba9-be50-38b9aada3f70 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:57.591393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887581b0-8d0a-4ba9-be50-38b9aada3f70 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:57.591672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887581b0-8d0a-4ba9-be50-38b9aada3f70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:57.592836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887581b0-8d0a-4ba9-be50-38b9aada3f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:57.594268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887581b0-8d0a-4ba9-be50-38b9aada3f70 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:57.595896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 10/18] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:35:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:35:57.601596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887581b0-8d0a-4ba9-be50-38b9aada3f70 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-3bc656d6-0dcd-4a90-abe8-0d9930e60c28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:35:57.603229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:35:57.604059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:35:57.604858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:35:57.604858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:35:57.604858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:35:57.604858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:35:57.604858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 9/19] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:35:57.610395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc656d6-0dcd-4a90-abe8-0d9930e60c28 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:35:57.665579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:35:57.682573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:35:57.701846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887581b0-8d0a-4ba9-be50-38b9aada3f70 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:57.708569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc656d6-0dcd-4a90-abe8-0d9930e60c28 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:35:57.735167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13e205-b7ee-490e-b8b2-44bb41732f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:57.735818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13e205-b7ee-490e-b8b2-44bb41732f67 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:57.736104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13e205-b7ee-490e-b8b2-44bb41732f67 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:57.736381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13e205-b7ee-490e-b8b2-44bb41732f67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:57.737209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13e205-b7ee-490e-b8b2-44bb41732f67 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.domain.id=default, user.id=38729f2fb8864984b35336009a142811) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:35:57.737517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13e205-b7ee-490e-b8b2-44bb41732f67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:57.738992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13e205-b7ee-490e-b8b2-44bb41732f67 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:57.742286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdab23a-0a36-4417-bb8f-155399fc23b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:35:57.742980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdab23a-0a36-4417-bb8f-155399fc23b0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:35:57.743253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdab23a-0a36-4417-bb8f-155399fc23b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:35:57.743534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdab23a-0a36-4417-bb8f-155399fc23b0 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:35:57.744259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdab23a-0a36-4417-bb8f-155399fc23b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:35:57.744570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdab23a-0a36-4417-bb8f-155399fc23b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:35:57.745588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 11/20] 10.222.0.62 () {56 vars in 1322 bytes} [Wed Jun 15 15:35:57 2022] GET /identity/v3/role_assignments?user.id=38729f2fb8864984b35336009a142811&scope.domain.id=default&role.id=433478cc318a467c931f13a757f161fa => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:35:57.745943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdab23a-0a36-4417-bb8f-155399fc23b0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:35:57.765525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 10/21] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:00.139838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae651590-81ad-4f67-8a93-ff3bf03df40b None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:00.140344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae651590-81ad-4f67-8a93-ff3bf03df40b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:00.140439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae651590-81ad-4f67-8a93-ff3bf03df40b None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:00.141767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 12/22] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:00.147167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265ce9b-297f-4ef3-bf74-e4f9eb3b4ad1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:00.147566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265ce9b-297f-4ef3-bf74-e4f9eb3b4ad1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:00.147731 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265ce9b-297f-4ef3-bf74-e4f9eb3b4ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:00.171936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8265ce9b-297f-4ef3-bf74-e4f9eb3b4ad1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:00.173260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8265ce9b-297f-4ef3-bf74-e4f9eb3b4ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:00.235827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa3cc2-0916-43c1-85b8-5dc71397b634 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:00.236450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa3cc2-0916-43c1-85b8-5dc71397b634 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:00.236918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa3cc2-0916-43c1-85b8-5dc71397b634 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:00.238715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 13/23] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:00.244033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e81a9b-a1f4-4163-93bc-41bb5cce9ebf None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:00.244561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e81a9b-a1f4-4163-93bc-41bb5cce9ebf None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:00.245031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e81a9b-a1f4-4163-93bc-41bb5cce9ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:00.267634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 11/24] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:00.291769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b2e81a9b-a1f4-4163-93bc-41bb5cce9ebf None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:00.293508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b2e81a9b-a1f4-4163-93bc-41bb5cce9ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:00.300764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265ce9b-297f-4ef3-bf74-e4f9eb3b4ad1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:00.409410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:00.428535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 14/25] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:00.469620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e81a9b-a1f4-4163-93bc-41bb5cce9ebf None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:00.544748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265ce9b-297f-4ef3-bf74-e4f9eb3b4ad1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:00.616922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:00.618577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c1560-dfba-436d-91da-569eb8537cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:00.619428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c1560-dfba-436d-91da-569eb8537cb5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:00.619720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c1560-dfba-436d-91da-569eb8537cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:00.620003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c1560-dfba-436d-91da-569eb8537cb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:00.651444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c1560-dfba-436d-91da-569eb8537cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-ed9c1560-dfba-436d-91da-569eb8537cb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:00.661490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:00.662400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:00.663636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 12/26] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:00.668806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c1560-dfba-436d-91da-569eb8537cb5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:00.680938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e81a9b-a1f4-4163-93bc-41bb5cce9ebf None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:00.779592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9a261-1134-4c16-a428-3e45234e9181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:00.780430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9a261-1134-4c16-a428-3e45234e9181 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:00.780711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9a261-1134-4c16-a428-3e45234e9181 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:00.780986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9a261-1134-4c16-a428-3e45234e9181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:00.797819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9a261-1134-4c16-a428-3e45234e9181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:00.799409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9a261-1134-4c16-a428-3e45234e9181 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-f4c9a261-1134-4c16-a428-3e45234e9181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:36:00.834615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:00.835505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:00.836312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:36:00.836312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:36:00.836312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:36:00.836312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:00.844497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 15/27] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:00.850114 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9a261-1134-4c16-a428-3e45234e9181 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:00.918795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c1560-dfba-436d-91da-569eb8537cb5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:00.943787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:00.960896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30190279-acbb-4e96-aafd-44f959e65357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:00.961987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30190279-acbb-4e96-aafd-44f959e65357 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:00.962905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30190279-acbb-4e96-aafd-44f959e65357 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:00.963468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30190279-acbb-4e96-aafd-44f959e65357 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:00.964695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30190279-acbb-4e96-aafd-44f959e65357 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:00.965387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30190279-acbb-4e96-aafd-44f959e65357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:00.967545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30190279-acbb-4e96-aafd-44f959e65357 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:00.981746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9a261-1134-4c16-a428-3e45234e9181 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:00.984048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 13/28] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:00.986265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf47242-70d5-4fb1-aee7-f44d1db43a0f None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:00.986634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf47242-70d5-4fb1-aee7-f44d1db43a0f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:00.987009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf47242-70d5-4fb1-aee7-f44d1db43a0f None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:00.988549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 14/29] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:00.989749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5892638-a829-4d85-99bf-a43dc87b5cd8 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:00.990618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5892638-a829-4d85-99bf-a43dc87b5cd8 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:00.991017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5892638-a829-4d85-99bf-a43dc87b5cd8 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:00.992298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 15/30] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:00.996634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5892638-a829-4d85-99bf-a43dc87b5cd8 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:01.015139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:01.026168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d75e0b-88e9-4cec-87c3-95826a584d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:01.027627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d75e0b-88e9-4cec-87c3-95826a584d53 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.028739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d75e0b-88e9-4cec-87c3-95826a584d53 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.029482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d75e0b-88e9-4cec-87c3-95826a584d53 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.030570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d75e0b-88e9-4cec-87c3-95826a584d53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:01.031076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d75e0b-88e9-4cec-87c3-95826a584d53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:01.033357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d75e0b-88e9-4cec-87c3-95826a584d53 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:01.062396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 16/31] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:01.063982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25461cb6-d3e5-4a2c-9174-a93386520c0c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.064484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25461cb6-d3e5-4a2c-9174-a93386520c0c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.064967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25461cb6-d3e5-4a2c-9174-a93386520c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.083759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-25461cb6-d3e5-4a2c-9174-a93386520c0c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:01.085329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-25461cb6-d3e5-4a2c-9174-a93386520c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:01.138087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5892638-a829-4d85-99bf-a43dc87b5cd8 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:01.160082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 17/32] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:01.162276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243bf86-a2f9-41c4-841f-9da4cf7530f0 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.162802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243bf86-a2f9-41c4-841f-9da4cf7530f0 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.163324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243bf86-a2f9-41c4-841f-9da4cf7530f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.179700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03931c2e-804c-43d6-b57c-c146be155a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:01.180875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03931c2e-804c-43d6-b57c-c146be155a53 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.181159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03931c2e-804c-43d6-b57c-c146be155a53 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.181434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03931c2e-804c-43d6-b57c-c146be155a53 None admin] PATH_INFO: `/v3/users/38729f2fb8864984b35336009a142811` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.183209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03931c2e-804c-43d6-b57c-c146be155a53 None admin] RBAC: Authorizing `identity:get_user(user_id=38729f2fb8864984b35336009a142811)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:01.185344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03931c2e-804c-43d6-b57c-c146be155a53 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:01.185994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-3243bf86-a2f9-41c4-841f-9da4cf7530f0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:01.187160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 16/33] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:00 2022] GET /identity/v3/users/38729f2fb8864984b35336009a142811 => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:01.187665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3243bf86-a2f9-41c4-841f-9da4cf7530f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:01.187952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03931c2e-804c-43d6-b57c-c146be155a53 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:01.236245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:01.283766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 18/34] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:01.290750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243bf86-a2f9-41c4-841f-9da4cf7530f0 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:01.331976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03931c2e-804c-43d6-b57c-c146be155a53 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:01.355496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:01.375905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2df1f-d28d-4c22-b619-5eb01ea9a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:01.376574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2df1f-d28d-4c22-b619-5eb01ea9a329 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.376855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2df1f-d28d-4c22-b619-5eb01ea9a329 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.377430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2df1f-d28d-4c22-b619-5eb01ea9a329 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.378561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2df1f-d28d-4c22-b619-5eb01ea9a329 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:01.380390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2df1f-d28d-4c22-b619-5eb01ea9a329 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:01.426008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243bf86-a2f9-41c4-841f-9da4cf7530f0 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:01.461890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e5177-133b-4dd0-9544-e2328b8af470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:01.462724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e5177-133b-4dd0-9544-e2328b8af470 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.463049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e5177-133b-4dd0-9544-e2328b8af470 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.463324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e5177-133b-4dd0-9544-e2328b8af470 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.464550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e5177-133b-4dd0-9544-e2328b8af470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:01.466635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e5177-133b-4dd0-9544-e2328b8af470 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:01.468363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 19/35] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:01.469100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e5177-133b-4dd0-9544-e2328b8af470 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:01.578771 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e5177-133b-4dd0-9544-e2328b8af470 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:01.595946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:01.619524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00064c15-16c8-4657-ac65-efc59d930f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:01.620196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00064c15-16c8-4657-ac65-efc59d930f13 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.620475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00064c15-16c8-4657-ac65-efc59d930f13 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.620753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00064c15-16c8-4657-ac65-efc59d930f13 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.621492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00064c15-16c8-4657-ac65-efc59d930f13 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:01.623500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00064c15-16c8-4657-ac65-efc59d930f13 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:01.665074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 17/36] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:36:01 2022] POST /identity/v3/projects => generated 301 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:01.666455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2df1f-d28d-4c22-b619-5eb01ea9a329 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:01.784981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:01.803667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2df1f-d28d-4c22-b619-5eb01ea9a329 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:01.844482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1ee5a-38d9-4889-8418-8b61f83f41bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:01.845573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1ee5a-38d9-4889-8418-8b61f83f41bd None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.846080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1ee5a-38d9-4889-8418-8b61f83f41bd None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.846565 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1ee5a-38d9-4889-8418-8b61f83f41bd None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.847574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1ee5a-38d9-4889-8418-8b61f83f41bd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:01.849325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1ee5a-38d9-4889-8418-8b61f83f41bd None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:01.865596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 18/37] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Jun 15 15:36:01 2022] POST /identity/v3/roles => generated 225 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:01.866760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1ee5a-38d9-4889-8418-8b61f83f41bd None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:01.894999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 20/38] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Jun 15 15:36:01 2022] POST /identity/v3/roles => generated 219 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:01.898546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4c5f5-1b40-49d4-9b3d-fb1c220d6780 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.898546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4c5f5-1b40-49d4-9b3d-fb1c220d6780 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.898546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4c5f5-1b40-49d4-9b3d-fb1c220d6780 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.898546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 21/39] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:01.899961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d3e5c-7c75-47a4-9446-576bda0eee99 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.900249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d3e5c-7c75-47a4-9446-576bda0eee99 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.900530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d3e5c-7c75-47a4-9446-576bda0eee99 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.901904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 22/40] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:01.904737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a878b66-e335-40cc-beea-1e20a9777dc1 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:01.905028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a878b66-e335-40cc-beea-1e20a9777dc1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:01.905305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a878b66-e335-40cc-beea-1e20a9777dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:01.925230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6a878b66-e335-40cc-beea-1e20a9777dc1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:01.926995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6a878b66-e335-40cc-beea-1e20a9777dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:02.011587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 23/41] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:02.013056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1856b-41c5-424a-9125-275af79d50fb None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:02.013349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1856b-41c5-424a-9125-275af79d50fb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:02.013636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1856b-41c5-424a-9125-275af79d50fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:02.022177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1ee5a-38d9-4889-8418-8b61f83f41bd None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:02.032329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6ed1856b-41c5-424a-9125-275af79d50fb None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:02.033558 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6ed1856b-41c5-424a-9125-275af79d50fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:02.072756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3a9b9-6382-4237-9828-06a2d0a7dabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:02.074127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3a9b9-6382-4237-9828-06a2d0a7dabf None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:02.074656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3a9b9-6382-4237-9828-06a2d0a7dabf None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:02.075355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3a9b9-6382-4237-9828-06a2d0a7dabf None admin] PATH_INFO: `/v3/domains/default/users/38729f2fb8864984b35336009a142811/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:02.079261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3a9b9-6382-4237-9828-06a2d0a7dabf None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=38729f2fb8864984b35336009a142811, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:02.080995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3a9b9-6382-4237-9828-06a2d0a7dabf None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:02.102119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 19/42] 10.222.0.62 () {58 vars in 1361 bytes} [Wed Jun 15 15:36:01 2022] PUT /identity/v3/domains/default/users/38729f2fb8864984b35336009a142811/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:02.103419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3a9b9-6382-4237-9828-06a2d0a7dabf None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:02.118853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:02.125648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 24/43] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:02.157092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1856b-41c5-424a-9125-275af79d50fb None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:02.199467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:02.257655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3a9b9-6382-4237-9828-06a2d0a7dabf None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:02.289618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1856b-41c5-424a-9125-275af79d50fb None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:02.298012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48965e4f-441b-4116-9b31-a5245d5aaf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:02.299129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48965e4f-441b-4116-9b31-a5245d5aaf3c None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:02.299129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48965e4f-441b-4116-9b31-a5245d5aaf3c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:02.299330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48965e4f-441b-4116-9b31-a5245d5aaf3c None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:02.302143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48965e4f-441b-4116-9b31-a5245d5aaf3c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:02.304360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48965e4f-441b-4116-9b31-a5245d5aaf3c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:02.320349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 20/44] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Jun 15 15:36:02 2022] POST /identity/v3/roles => generated 223 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:02.330045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b04039-6a8c-4a14-91eb-849309f70c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:02.330919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b04039-6a8c-4a14-91eb-849309f70c81 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:02.331210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b04039-6a8c-4a14-91eb-849309f70c81 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:02.331493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b04039-6a8c-4a14-91eb-849309f70c81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:02.332744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b04039-6a8c-4a14-91eb-849309f70c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:02.334383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b04039-6a8c-4a14-91eb-849309f70c81 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:02.336133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 25/45] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:02.342764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48965e4f-441b-4116-9b31-a5245d5aaf3c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:02.437872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:02.492897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48965e4f-441b-4116-9b31-a5245d5aaf3c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:02.530822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a547e0-c22b-453f-a7f5-e042d802f0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:02.531147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a547e0-c22b-453f-a7f5-e042d802f0e5 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:02.531709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a547e0-c22b-453f-a7f5-e042d802f0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:02.531709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a547e0-c22b-453f-a7f5-e042d802f0e5 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:02.533045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a547e0-c22b-453f-a7f5-e042d802f0e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:02.533889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a547e0-c22b-453f-a7f5-e042d802f0e5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:02.566301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 21/46] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:36:02 2022] POST /identity/v3/projects => generated 312 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:04.274873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0e6d6-c5ca-4d32-8f99-e1e8a4134440 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:04.275391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0e6d6-c5ca-4d32-8f99-e1e8a4134440 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:04.275475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0e6d6-c5ca-4d32-8f99-e1e8a4134440 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:04.276825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 26/47] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:04.282252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add397ff-e811-4120-828e-401675fb8df1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:04.282630 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add397ff-e811-4120-828e-401675fb8df1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:04.282869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add397ff-e811-4120-828e-401675fb8df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:04.299410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-add397ff-e811-4120-828e-401675fb8df1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:04.300938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-add397ff-e811-4120-828e-401675fb8df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:04.368689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 22/48] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:04.391935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0e6d6-c5ca-4d32-8f99-e1e8a4134440 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:04.497780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0e6d6-c5ca-4d32-8f99-e1e8a4134440 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:04.499362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:04.532507 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c22dbc-eae9-49b7-a856-32716fc5caeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:04.533230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c22dbc-eae9-49b7-a856-32716fc5caeb None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:04.533517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c22dbc-eae9-49b7-a856-32716fc5caeb None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:04.533801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c22dbc-eae9-49b7-a856-32716fc5caeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:04.544330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c22dbc-eae9-49b7-a856-32716fc5caeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-e4c22dbc-eae9-49b7-a856-32716fc5caeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:04.552057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:04.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:04.559044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 27/49] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:04.559833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add397ff-e811-4120-828e-401675fb8df1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:04.607545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:04.662215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add397ff-e811-4120-828e-401675fb8df1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:04.685264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dfaec-0bc0-4c02-a1e2-2215e02240b9 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:04.685646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dfaec-0bc0-4c02-a1e2-2215e02240b9 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:04.685996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dfaec-0bc0-4c02-a1e2-2215e02240b9 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:04.687405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 28/50] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:36:04.692237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0834ce-4ef5-400c-9ace-3c0c570b4fed None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:04.692531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0834ce-4ef5-400c-9ace-3c0c570b4fed None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:04.692811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0834ce-4ef5-400c-9ace-3c0c570b4fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:04.696654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecadaf-b1fb-4d01-b896-cf3c4d6a2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:04.697329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecadaf-b1fb-4d01-b896-cf3c4d6a2a94 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:04.697622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecadaf-b1fb-4d01-b896-cf3c4d6a2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:04.697905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecadaf-b1fb-4d01-b896-cf3c4d6a2a94 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:04.698663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecadaf-b1fb-4d01-b896-cf3c4d6a2a94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:04.699005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecadaf-b1fb-4d01-b896-cf3c4d6a2a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:04.700906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecadaf-b1fb-4d01-b896-cf3c4d6a2a94 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:04.709119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 23/51] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:04.710227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ef0834ce-4ef5-400c-9ace-3c0c570b4fed None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:04.711453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ef0834ce-4ef5-400c-9ace-3c0c570b4fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:04.714486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecadaf-b1fb-4d01-b896-cf3c4d6a2a94 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:04.778739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 29/52] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:04.802334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0834ce-4ef5-400c-9ace-3c0c570b4fed None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:04.813491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecadaf-b1fb-4d01-b896-cf3c4d6a2a94 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:04.827682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:04.850405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03ef85-96fd-4d2e-9a61-26e4175b6549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:04.851318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03ef85-96fd-4d2e-9a61-26e4175b6549 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:04.851607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03ef85-96fd-4d2e-9a61-26e4175b6549 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:04.851889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03ef85-96fd-4d2e-9a61-26e4175b6549 None admin] PATH_INFO: `/v3/users/38729f2fb8864984b35336009a142811` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:04.853093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03ef85-96fd-4d2e-9a61-26e4175b6549 None admin] RBAC: Authorizing `identity:get_user(user_id=38729f2fb8864984b35336009a142811)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:04.854548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03ef85-96fd-4d2e-9a61-26e4175b6549 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:04.856778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 24/53] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:04 2022] GET /identity/v3/users/38729f2fb8864984b35336009a142811 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:04.862078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03ef85-96fd-4d2e-9a61-26e4175b6549 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:04.890878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:04.900120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0834ce-4ef5-400c-9ace-3c0c570b4fed None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:04.932905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8342bf-0340-4d07-bfbd-65b951b08e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:04.933695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8342bf-0340-4d07-bfbd-65b951b08e8b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:04.933976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8342bf-0340-4d07-bfbd-65b951b08e8b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:04.934784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8342bf-0340-4d07-bfbd-65b951b08e8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:04.935956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8342bf-0340-4d07-bfbd-65b951b08e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:04.937407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8342bf-0340-4d07-bfbd-65b951b08e8b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:04.939038 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 30/54] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:04.944485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8342bf-0340-4d07-bfbd-65b951b08e8b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:04.957802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03ef85-96fd-4d2e-9a61-26e4175b6549 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:04.993594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a539-ca0d-4d49-bee0-c72a1a167f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:04.994382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a539-ca0d-4d49-bee0-c72a1a167f41 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:04.995119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a539-ca0d-4d49-bee0-c72a1a167f41 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:04.995410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a539-ca0d-4d49-bee0-c72a1a167f41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:04.997124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55a539-ca0d-4d49-bee0-c72a1a167f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:04.998558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55a539-ca0d-4d49-bee0-c72a1a167f41 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:05.000199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 25/55] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:05.005781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a539-ca0d-4d49-bee0-c72a1a167f41 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:05.040064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8342bf-0340-4d07-bfbd-65b951b08e8b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:05.053878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:05.073587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c71135-46b7-4c1f-b8fe-710a44a7da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:05.074251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71135-46b7-4c1f-b8fe-710a44a7da87 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:05.074534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71135-46b7-4c1f-b8fe-710a44a7da87 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:05.074890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71135-46b7-4c1f-b8fe-710a44a7da87 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:05.075591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c71135-46b7-4c1f-b8fe-710a44a7da87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:05.077045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c71135-46b7-4c1f-b8fe-710a44a7da87 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:05.102864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 31/56] 10.222.0.62 () {60 vars in 1140 bytes} [Wed Jun 15 15:36:04 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:36:05.111743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a539-ca0d-4d49-bee0-c72a1a167f41 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:05.147896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4915b-b6c1-4a9f-a790-979a058f1754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:05.148571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4915b-b6c1-4a9f-a790-979a058f1754 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:05.149111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4915b-b6c1-4a9f-a790-979a058f1754 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:05.149387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4915b-b6c1-4a9f-a790-979a058f1754 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:05.150253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4915b-b6c1-4a9f-a790-979a058f1754 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.domain.id=default, user.id=38729f2fb8864984b35336009a142811) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:05.150600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4915b-b6c1-4a9f-a790-979a058f1754 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:05.152165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4915b-b6c1-4a9f-a790-979a058f1754 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:05.162257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 26/57] 10.222.0.62 () {56 vars in 1322 bytes} [Wed Jun 15 15:36:05 2022] GET /identity/v3/role_assignments?user.id=38729f2fb8864984b35336009a142811&scope.domain.id=default&role.id=433478cc318a467c931f13a757f161fa => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:07.236155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba34d7-e753-4819-98b4-fab6a305a4b1 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:07.236666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba34d7-e753-4819-98b4-fab6a305a4b1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:07.236751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba34d7-e753-4819-98b4-fab6a305a4b1 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:07.238082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 32/58] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:07.243173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d19dc1-4828-43cd-8b97-9990acdd9270 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:07.243580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d19dc1-4828-43cd-8b97-9990acdd9270 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:07.243746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d19dc1-4828-43cd-8b97-9990acdd9270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:07.260588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-25d19dc1-4828-43cd-8b97-9990acdd9270 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:07.261782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-25d19dc1-4828-43cd-8b97-9990acdd9270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:07.327155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 27/59] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:07.351925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba34d7-e753-4819-98b4-fab6a305a4b1 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:07.366852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:07.453405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba34d7-e753-4819-98b4-fab6a305a4b1 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:07.485270 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bceb8-c6ac-431c-abe0-2b2c66a76884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:07.486051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bceb8-c6ac-431c-abe0-2b2c66a76884 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:07.486384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bceb8-c6ac-431c-abe0-2b2c66a76884 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:07.486612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bceb8-c6ac-431c-abe0-2b2c66a76884 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:07.487802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bceb8-c6ac-431c-abe0-2b2c66a76884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:07.489243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bceb8-c6ac-431c-abe0-2b2c66a76884 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:07.490860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 33/60] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:07.496321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d19dc1-4828-43cd-8b97-9990acdd9270 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:07.537245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:07.592370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d19dc1-4828-43cd-8b97-9990acdd9270 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:07.626273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de31ae-8b66-486e-981e-cedc4dccb16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:07.626974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de31ae-8b66-486e-981e-cedc4dccb16c None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:07.627265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de31ae-8b66-486e-981e-cedc4dccb16c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:07.627551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de31ae-8b66-486e-981e-cedc4dccb16c None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:07.628261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de31ae-8b66-486e-981e-cedc4dccb16c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:07.629712 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de31ae-8b66-486e-981e-cedc4dccb16c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:07.658777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 28/61] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:36:07 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:10.973527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705d265-e8f4-4454-9ba9-40e2e1c87486 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:10.974410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705d265-e8f4-4454-9ba9-40e2e1c87486 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:10.974869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705d265-e8f4-4454-9ba9-40e2e1c87486 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:10.976399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 34/62] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:10.982314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921aee8-0588-4128-8847-07a455f11ecc None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:10.983114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921aee8-0588-4128-8847-07a455f11ecc None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:10.983552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921aee8-0588-4128-8847-07a455f11ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.007070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-3921aee8-0588-4128-8847-07a455f11ecc None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:11.008550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3921aee8-0588-4128-8847-07a455f11ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:11.009147 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a16f98-45a3-4794-9497-7879df0667bb None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.009680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a16f98-45a3-4794-9497-7879df0667bb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.010195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a16f98-45a3-4794-9497-7879df0667bb None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.011946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 35/63] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:11.017822 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada20f5-5da3-4c27-b50a-41639732f441 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.018439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada20f5-5da3-4c27-b50a-41639732f441 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.018989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada20f5-5da3-4c27-b50a-41639732f441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.051712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-cada20f5-5da3-4c27-b50a-41639732f441 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:11.052922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cada20f5-5da3-4c27-b50a-41639732f441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:11.114038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 29/64] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:11.115786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74fba9-8278-45f1-9c5d-82c3611893a5 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.116279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74fba9-8278-45f1-9c5d-82c3611893a5 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.116784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74fba9-8278-45f1-9c5d-82c3611893a5 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.118409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 30/65] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:11.124320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d0b6-9160-43fb-b152-84e0ba58d588 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.124954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d0b6-9160-43fb-b152-84e0ba58d588 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.125302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d0b6-9160-43fb-b152-84e0ba58d588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.146412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 36/66] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:11.154773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada20f5-5da3-4c27-b50a-41639732f441 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:11.154895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-07a9d0b6-9160-43fb-b152-84e0ba58d588 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:11.154970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-07a9d0b6-9160-43fb-b152-84e0ba58d588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:11.181940 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:11.191093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:11.238949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 31/67] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:11.240114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9d0b6-9160-43fb-b152-84e0ba58d588 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:11.299995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada20f5-5da3-4c27-b50a-41639732f441 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:11.347149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb8961-05cf-4633-9809-1425ceff8937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:11.348311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb8961-05cf-4633-9809-1425ceff8937 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.348814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb8961-05cf-4633-9809-1425ceff8937 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.349286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb8961-05cf-4633-9809-1425ceff8937 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.360239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb8961-05cf-4633-9809-1425ceff8937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-ebdb8961-05cf-4633-9809-1425ceff8937 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:11.368264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:11.369200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:11.369967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:11.371881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 37/68] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:11.372444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9d0b6-9160-43fb-b152-84e0ba58d588 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:11.373042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb8961-05cf-4633-9809-1425ceff8937 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:11.411906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e4d6a-0fef-4f59-97f8-f0a24bdbcf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:11.412872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e4d6a-0fef-4f59-97f8-f0a24bdbcf18 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.413402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e4d6a-0fef-4f59-97f8-f0a24bdbcf18 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.413906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e4d6a-0fef-4f59-97f8-f0a24bdbcf18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.425407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e4d6a-0fef-4f59-97f8-f0a24bdbcf18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-df3e4d6a-0fef-4f59-97f8-f0a24bdbcf18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:11.435905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:11.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:11.437692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:11.438574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 32/69] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:11.443883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f30334-9eb6-413a-ad95-3ff438c4993e None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.443883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f30334-9eb6-413a-ad95-3ff438c4993e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.443883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f30334-9eb6-413a-ad95-3ff438c4993e None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.443883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 33/70] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:11.443883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f30334-9eb6-413a-ad95-3ff438c4993e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:11.461984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:11.470628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:11.497225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb8961-05cf-4633-9809-1425ceff8937 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:11.532966 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06ee1-bf63-49b5-b847-9756986993b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:11.533813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06ee1-bf63-49b5-b847-9756986993b4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.534103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06ee1-bf63-49b5-b847-9756986993b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.534394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06ee1-bf63-49b5-b847-9756986993b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.546289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06ee1-bf63-49b5-b847-9756986993b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-82c06ee1-bf63-49b5-b847-9756986993b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:11.554645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:11.555738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:11.556483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 38/71] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:11.559979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06ee1-bf63-49b5-b847-9756986993b4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:11.574085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f30334-9eb6-413a-ad95-3ff438c4993e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:11.612407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e759b0-8039-463b-916f-5325f1a936f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:11.613082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e759b0-8039-463b-916f-5325f1a936f0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.613364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e759b0-8039-463b-916f-5325f1a936f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.613646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e759b0-8039-463b-916f-5325f1a936f0 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.614398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e759b0-8039-463b-916f-5325f1a936f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:11.614708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e759b0-8039-463b-916f-5325f1a936f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:11.616127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e759b0-8039-463b-916f-5325f1a936f0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:11.625366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 34/72] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:11.626560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d7e5e-20fc-4bb8-a9ec-4c08e1a24655 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.626879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d7e5e-20fc-4bb8-a9ec-4c08e1a24655 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.627162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d7e5e-20fc-4bb8-a9ec-4c08e1a24655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.644933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b05d7e5e-20fc-4bb8-a9ec-4c08e1a24655 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:11.646577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b05d7e5e-20fc-4bb8-a9ec-4c08e1a24655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:11.678967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06ee1-bf63-49b5-b847-9756986993b4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:11.706046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:11.711943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620fad8-9cd9-45d8-8304-9cf0dafeb31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:11.712615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620fad8-9cd9-45d8-8304-9cf0dafeb31e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.712906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620fad8-9cd9-45d8-8304-9cf0dafeb31e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.713034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 35/73] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:11.726793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620fad8-9cd9-45d8-8304-9cf0dafeb31e None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.726793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620fad8-9cd9-45d8-8304-9cf0dafeb31e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:11.726978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d7e5e-20fc-4bb8-a9ec-4c08e1a24655 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:11.727066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620fad8-9cd9-45d8-8304-9cf0dafeb31e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:11.727066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620fad8-9cd9-45d8-8304-9cf0dafeb31e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:11.727209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:11.741209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 39/74] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:11.742596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da724c-020e-449d-9dda-fedc950715c3 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.742931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da724c-020e-449d-9dda-fedc950715c3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.743216 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da724c-020e-449d-9dda-fedc950715c3 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.744469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 40/75] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:11.745754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da724c-020e-449d-9dda-fedc950715c3 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:11.816398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d7e5e-20fc-4bb8-a9ec-4c08e1a24655 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:11.845723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da724c-020e-449d-9dda-fedc950715c3 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:11.849925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cc7c7-6e73-45c3-9a7c-cafae97acd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:11.850596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cc7c7-6e73-45c3-9a7c-cafae97acd60 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.850922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cc7c7-6e73-45c3-9a7c-cafae97acd60 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.851205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cc7c7-6e73-45c3-9a7c-cafae97acd60 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.851955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54cc7c7-6e73-45c3-9a7c-cafae97acd60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:11.852265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54cc7c7-6e73-45c3-9a7c-cafae97acd60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:11.853655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54cc7c7-6e73-45c3-9a7c-cafae97acd60 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:11.861773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 36/76] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:11.866542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cc7c7-6e73-45c3-9a7c-cafae97acd60 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:11.878985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e56e9-284c-424d-b6a7-902da528a52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:11.879857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e56e9-284c-424d-b6a7-902da528a52a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:11.880139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e56e9-284c-424d-b6a7-902da528a52a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:11.880420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e56e9-284c-424d-b6a7-902da528a52a None admin] PATH_INFO: `/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:11.894266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e56e9-284c-424d-b6a7-902da528a52a None admin] RBAC: Authorizing `identity:get_user(user_id=4042a3d4d3d8483b9f9a6f1c750f1360)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:11.895780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e56e9-284c-424d-b6a7-902da528a52a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:11.897518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 41/77] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:11 2022] GET /identity/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:11.898292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e56e9-284c-424d-b6a7-902da528a52a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:11.937407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:11.946999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:11.967678 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cc7c7-6e73-45c3-9a7c-cafae97acd60 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:11.998714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e56e9-284c-424d-b6a7-902da528a52a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.001312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a669c-560e-47b2-b456-de723644ba06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.002106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a669c-560e-47b2-b456-de723644ba06 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.002388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a669c-560e-47b2-b456-de723644ba06 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.002668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a669c-560e-47b2-b456-de723644ba06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.012607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a669c-560e-47b2-b456-de723644ba06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-cd7a669c-560e-47b2-b456-de723644ba06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:12.020246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:12.021134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:12.021974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 37/78] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:12.024181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f6f83-2ddc-46b2-bae8-ac13be6d643c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.024494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f6f83-2ddc-46b2-bae8-ac13be6d643c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.024753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f6f83-2ddc-46b2-bae8-ac13be6d643c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.033037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4c187-d02f-4c77-abf3-5643f192c7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.033906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4c187-d02f-4c77-abf3-5643f192c7c8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.034193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4c187-d02f-4c77-abf3-5643f192c7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.034473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4c187-d02f-4c77-abf3-5643f192c7c8 None admin] PATH_INFO: `/v3/users/38729f2fb8864984b35336009a142811` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.035700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4c187-d02f-4c77-abf3-5643f192c7c8 None admin] RBAC: Authorizing `identity:get_user(user_id=38729f2fb8864984b35336009a142811)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.037151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4c187-d02f-4c77-abf3-5643f192c7c8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:12.038932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 42/79] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:11 2022] GET /identity/v3/users/38729f2fb8864984b35336009a142811 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:12.039548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4c187-d02f-4c77-abf3-5643f192c7c8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.040644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d51f6f83-2ddc-46b2-bae8-ac13be6d643c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:12.041845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d51f6f83-2ddc-46b2-bae8-ac13be6d643c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:12.107379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 38/80] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:12.108167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f6f83-2ddc-46b2-bae8-ac13be6d643c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.148072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4c187-d02f-4c77-abf3-5643f192c7c8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.165180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:12.168479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:12.182662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ef9f9-ca76-43f7-bc82-e92df57d0aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.183617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ef9f9-ca76-43f7-bc82-e92df57d0aa4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.184410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ef9f9-ca76-43f7-bc82-e92df57d0aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.184692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ef9f9-ca76-43f7-bc82-e92df57d0aa4 None admin] PATH_INFO: `/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.185903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ef9f9-ca76-43f7-bc82-e92df57d0aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=4042a3d4d3d8483b9f9a6f1c750f1360)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.187385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ef9f9-ca76-43f7-bc82-e92df57d0aa4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:12.189007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 43/81] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:12 2022] GET /identity/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:12.193830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ef9f9-ca76-43f7-bc82-e92df57d0aa4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.209759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f6f83-2ddc-46b2-bae8-ac13be6d643c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.242939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331264-b1f3-41be-b034-8d6a65d6a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.243586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331264-b1f3-41be-b034-8d6a65d6a752 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.243870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331264-b1f3-41be-b034-8d6a65d6a752 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.244147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331264-b1f3-41be-b034-8d6a65d6a752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.285152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ef9f9-ca76-43f7-bc82-e92df57d0aa4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.309895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a331264-b1f3-41be-b034-8d6a65d6a752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.311380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a331264-b1f3-41be-b034-8d6a65d6a752 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:12.319728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f4377-546e-483e-93d0-b02ec9288efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.320393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f4377-546e-483e-93d0-b02ec9288efd None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.320676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f4377-546e-483e-93d0-b02ec9288efd None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.320954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f4377-546e-483e-93d0-b02ec9288efd None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.321698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f4377-546e-483e-93d0-b02ec9288efd None admin] RBAC: Adding query filter params (name=member) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:12.322004 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f4377-546e-483e-93d0-b02ec9288efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.323416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f4377-546e-483e-93d0-b02ec9288efd None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:12.331367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 44/82] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:12.332144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f4377-546e-483e-93d0-b02ec9288efd None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.370851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:12.381292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:12.409906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 39/83] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:12.410720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331264-b1f3-41be-b034-8d6a65d6a752 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.438674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f4377-546e-483e-93d0-b02ec9288efd None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.472641 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e7c95-73b2-4830-a413-cf0fc5309a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.473294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e7c95-73b2-4830-a413-cf0fc5309a9d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.473579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e7c95-73b2-4830-a413-cf0fc5309a9d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.473858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e7c95-73b2-4830-a413-cf0fc5309a9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.507971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331264-b1f3-41be-b034-8d6a65d6a752 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.539561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e7c95-73b2-4830-a413-cf0fc5309a9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.541033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e7c95-73b2-4830-a413-cf0fc5309a9d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:12.541568 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857115e-80dc-42f3-8634-67fbabcf1491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.542355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857115e-80dc-42f3-8634-67fbabcf1491 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.542644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857115e-80dc-42f3-8634-67fbabcf1491 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.543473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857115e-80dc-42f3-8634-67fbabcf1491 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.544676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857115e-80dc-42f3-8634-67fbabcf1491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.546122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857115e-80dc-42f3-8634-67fbabcf1491 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:12.547797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 40/84] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:12.548563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857115e-80dc-42f3-8634-67fbabcf1491 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.580858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:12.586865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:12.641157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 45/85] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:12.641920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e7c95-73b2-4830-a413-cf0fc5309a9d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.651099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857115e-80dc-42f3-8634-67fbabcf1491 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.684730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38db3dd-5fac-4cea-af29-52cf622e1cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.685369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38db3dd-5fac-4cea-af29-52cf622e1cb7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.685659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38db3dd-5fac-4cea-af29-52cf622e1cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.685938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38db3dd-5fac-4cea-af29-52cf622e1cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.744025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e7c95-73b2-4830-a413-cf0fc5309a9d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.756784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38db3dd-5fac-4cea-af29-52cf622e1cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.758260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38db3dd-5fac-4cea-af29-52cf622e1cb7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:12.778382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354708bb-2080-40c8-a5cd-f49e62211677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.779279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354708bb-2080-40c8-a5cd-f49e62211677 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.779561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354708bb-2080-40c8-a5cd-f49e62211677 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.779837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354708bb-2080-40c8-a5cd-f49e62211677 None admin] PATH_INFO: `/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.781030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354708bb-2080-40c8-a5cd-f49e62211677 None admin] RBAC: Authorizing `identity:get_user(user_id=4042a3d4d3d8483b9f9a6f1c750f1360)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.782488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354708bb-2080-40c8-a5cd-f49e62211677 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:12.784133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 46/86] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:12 2022] GET /identity/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:12.784892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354708bb-2080-40c8-a5cd-f49e62211677 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.790443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:12.802607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:12.859747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 41/87] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:12.860739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38db3dd-5fac-4cea-af29-52cf622e1cb7 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.886492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354708bb-2080-40c8-a5cd-f49e62211677 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.920187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7763a0-da9e-4a19-a573-79e7211d20b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.920971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7763a0-da9e-4a19-a573-79e7211d20b4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.921251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7763a0-da9e-4a19-a573-79e7211d20b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.921524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7763a0-da9e-4a19-a573-79e7211d20b4 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.922689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7763a0-da9e-4a19-a573-79e7211d20b4 None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.924155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7763a0-da9e-4a19-a573-79e7211d20b4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:12.925908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 47/88] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:12 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:12.926579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7763a0-da9e-4a19-a573-79e7211d20b4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:12.958459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38db3dd-5fac-4cea-af29-52cf622e1cb7 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:12.986194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:12.992643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3755e4-105b-43fd-acd5-3528049dac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:12.993315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3755e4-105b-43fd-acd5-3528049dac57 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:12.993605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3755e4-105b-43fd-acd5-3528049dac57 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:12.993889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3755e4-105b-43fd-acd5-3528049dac57 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:12.994612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3755e4-105b-43fd-acd5-3528049dac57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:12.996118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3755e4-105b-43fd-acd5-3528049dac57 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:13.007411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:13.015523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 42/89] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:36:12 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:13.015523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3755e4-105b-43fd-acd5-3528049dac57 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:13.027982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7763a0-da9e-4a19-a573-79e7211d20b4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:13.065961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dae44-3da8-4a81-abf7-4f7d4efcf32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:13.066889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dae44-3da8-4a81-abf7-4f7d4efcf32d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:13.067182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dae44-3da8-4a81-abf7-4f7d4efcf32d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:13.067464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dae44-3da8-4a81-abf7-4f7d4efcf32d None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:13.068846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dae44-3da8-4a81-abf7-4f7d4efcf32d None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:13.070750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dae44-3da8-4a81-abf7-4f7d4efcf32d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:13.073325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 48/90] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:12 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:13.074220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dae44-3da8-4a81-abf7-4f7d4efcf32d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:13.136080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3755e4-105b-43fd-acd5-3528049dac57 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:13.179139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b83b9-d142-4621-9d82-fe6305b09117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:13.179721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b83b9-d142-4621-9d82-fe6305b09117 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:13.180362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b83b9-d142-4621-9d82-fe6305b09117 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:13.180362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b83b9-d142-4621-9d82-fe6305b09117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:13.190523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dae44-3da8-4a81-abf7-4f7d4efcf32d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:13.228243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979211e-49fe-46c9-bc2d-470b6af5e201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:13.229163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979211e-49fe-46c9-bc2d-470b6af5e201 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:13.229441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979211e-49fe-46c9-bc2d-470b6af5e201 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:13.229726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979211e-49fe-46c9-bc2d-470b6af5e201 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:13.231521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979211e-49fe-46c9-bc2d-470b6af5e201 None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:13.232985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979211e-49fe-46c9-bc2d-470b6af5e201 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:13.234698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 49/91] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:13 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:13.239723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979211e-49fe-46c9-bc2d-470b6af5e201 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:13.246584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:13.265666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b83b9-d142-4621-9d82-fe6305b09117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:13.266615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:13.267505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b83b9-d142-4621-9d82-fe6305b09117 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:13.356684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979211e-49fe-46c9-bc2d-470b6af5e201 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:13.396863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 43/92] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:13.398561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b83b9-d142-4621-9d82-fe6305b09117 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:13.410377 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37c9c2-7be4-4dd6-830e-a224b1ea2cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:13.411160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37c9c2-7be4-4dd6-830e-a224b1ea2cbb None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:13.411445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37c9c2-7be4-4dd6-830e-a224b1ea2cbb None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:13.411722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37c9c2-7be4-4dd6-830e-a224b1ea2cbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:13.412898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37c9c2-7be4-4dd6-830e-a224b1ea2cbb None admin] RBAC: Adding query filter params (role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=54d37e7eb5474cf387beec6e56303b4f, user.id=4042a3d4d3d8483b9f9a6f1c750f1360) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:13.413501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37c9c2-7be4-4dd6-830e-a224b1ea2cbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:13.415260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37c9c2-7be4-4dd6-830e-a224b1ea2cbb None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:13.423605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 50/93] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:13 2022] GET /identity/v3/role_assignments?user.id=4042a3d4d3d8483b9f9a6f1c750f1360&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:13.424235 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37c9c2-7be4-4dd6-830e-a224b1ea2cbb None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:13.532986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:13.562274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b83b9-d142-4621-9d82-fe6305b09117 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:13.594232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:13.617881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37c9c2-7be4-4dd6-830e-a224b1ea2cbb None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:13.623344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b0021-24b9-4de4-8bcf-3b3b5ef824ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:13.624231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b0021-24b9-4de4-8bcf-3b3b5ef824ce None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:13.624720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b0021-24b9-4de4-8bcf-3b3b5ef824ce None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:13.626021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b0021-24b9-4de4-8bcf-3b3b5ef824ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:13.627095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b0021-24b9-4de4-8bcf-3b3b5ef824ce None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=54d37e7eb5474cf387beec6e56303b4f, user.id=38729f2fb8864984b35336009a142811) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:13.628039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b0021-24b9-4de4-8bcf-3b3b5ef824ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:13.630047 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b0021-24b9-4de4-8bcf-3b3b5ef824ce None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:13.643542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 44/94] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:13 2022] GET /identity/v3/role_assignments?user.id=38729f2fb8864984b35336009a142811&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=433478cc318a467c931f13a757f161fa => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:13.645046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b0021-24b9-4de4-8bcf-3b3b5ef824ce None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:13.711613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dec0c-62eb-474c-a0a0-26a08109c260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:13.712366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dec0c-62eb-474c-a0a0-26a08109c260 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:13.712725 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dec0c-62eb-474c-a0a0-26a08109c260 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:13.713099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dec0c-62eb-474c-a0a0-26a08109c260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:13.714013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933dec0c-62eb-474c-a0a0-26a08109c260 None admin] RBAC: Adding query filter params (role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=54d37e7eb5474cf387beec6e56303b4f, user.id=4042a3d4d3d8483b9f9a6f1c750f1360) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:13.714407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933dec0c-62eb-474c-a0a0-26a08109c260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:13.716591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933dec0c-62eb-474c-a0a0-26a08109c260 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:13.729761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 51/95] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:13 2022] GET /identity/v3/role_assignments?user.id=4042a3d4d3d8483b9f9a6f1c750f1360&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:13.798986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b0021-24b9-4de4-8bcf-3b3b5ef824ce None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:13.858021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947ac6a-3e4f-4f4b-a4e3-3c89fb68fb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:13.858953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947ac6a-3e4f-4f4b-a4e3-3c89fb68fb0b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:13.859243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947ac6a-3e4f-4f4b-a4e3-3c89fb68fb0b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:13.859528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947ac6a-3e4f-4f4b-a4e3-3c89fb68fb0b None admin] PATH_INFO: `/v3/projects/0e9d0a88b7cf41aa8d2ad10351114364` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:13.860848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947ac6a-3e4f-4f4b-a4e3-3c89fb68fb0b None admin] RBAC: Authorizing `identity:get_project(project_id=0e9d0a88b7cf41aa8d2ad10351114364)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:13.862331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947ac6a-3e4f-4f4b-a4e3-3c89fb68fb0b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:13.864145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 45/96] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:13 2022] GET /identity/v3/projects/0e9d0a88b7cf41aa8d2ad10351114364 => generated 312 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:13.870446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dec0c-62eb-474c-a0a0-26a08109c260 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:14.026146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dec0c-62eb-474c-a0a0-26a08109c260 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:14.077927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f9967-d11a-4fdc-88fb-d56832f87cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:14.078970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f9967-d11a-4fdc-88fb-d56832f87cee None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:14.079460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f9967-d11a-4fdc-88fb-d56832f87cee None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:14.079938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f9967-d11a-4fdc-88fb-d56832f87cee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:14.080976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f9967-d11a-4fdc-88fb-d56832f87cee None admin] RBAC: Adding query filter params (role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=0e9d0a88b7cf41aa8d2ad10351114364, user.id=4042a3d4d3d8483b9f9a6f1c750f1360) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:14.081519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f9967-d11a-4fdc-88fb-d56832f87cee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:14.083236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f9967-d11a-4fdc-88fb-d56832f87cee None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:14.095344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 52/97] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:13 2022] GET /identity/v3/role_assignments?user.id=4042a3d4d3d8483b9f9a6f1c750f1360&scope.project.id=0e9d0a88b7cf41aa8d2ad10351114364&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:16.064286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfab7fd-e2e5-4ef6-90c3-cae704cd98eb None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.065129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfab7fd-e2e5-4ef6-90c3-cae704cd98eb None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.065556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfab7fd-e2e5-4ef6-90c3-cae704cd98eb None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.067097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 46/98] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:16.080166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e813b-63a7-49a4-ad29-9183f6f9d054 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.080621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e813b-63a7-49a4-ad29-9183f6f9d054 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.080722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e813b-63a7-49a4-ad29-9183f6f9d054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.083406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165ecea-ac1f-4272-bc5f-b9cb1a06e466 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.083885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165ecea-ac1f-4272-bc5f-b9cb1a06e466 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.084342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165ecea-ac1f-4272-bc5f-b9cb1a06e466 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.085960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 47/99] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:36:16.091061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8950e-6c73-4787-9de1-e7395c7ce594 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.091562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8950e-6c73-4787-9de1-e7395c7ce594 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.092017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8950e-6c73-4787-9de1-e7395c7ce594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.107735 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-da6e813b-63a7-49a4-ad29-9183f6f9d054 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:16.109293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-da6e813b-63a7-49a4-ad29-9183f6f9d054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:16.116025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-13c8950e-6c73-4787-9de1-e7395c7ce594 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:16.117226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-13c8950e-6c73-4787-9de1-e7395c7ce594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:16.118545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:16.145270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:16.255076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 48/100] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:16.256711 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21715cd-7d4e-4484-9293-37d12a8a8743 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.257005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21715cd-7d4e-4484-9293-37d12a8a8743 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.257288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21715cd-7d4e-4484-9293-37d12a8a8743 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.259195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 49/101] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:16.273486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 53/102] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:16.292253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc4c1f-b3e2-45ea-9b82-acd833454e86 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.292542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc4c1f-b3e2-45ea-9b82-acd833454e86 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.293417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc4c1f-b3e2-45ea-9b82-acd833454e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.321545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e813b-63a7-49a4-ad29-9183f6f9d054 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:16.332626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-95bc4c1f-b3e2-45ea-9b82-acd833454e86 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:16.333886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-95bc4c1f-b3e2-45ea-9b82-acd833454e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:16.463915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 50/103] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:16.465362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc4c1f-b3e2-45ea-9b82-acd833454e86 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:16.522382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e813b-63a7-49a4-ad29-9183f6f9d054 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:16.593244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8609a-42f2-4af0-86cd-28a242945e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:16.593980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8609a-42f2-4af0-86cd-28a242945e67 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.594268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8609a-42f2-4af0-86cd-28a242945e67 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.594548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8609a-42f2-4af0-86cd-28a242945e67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.618077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:16.626159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8609a-42f2-4af0-86cd-28a242945e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-05f8609a-42f2-4af0-86cd-28a242945e67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:16.634211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:16.635206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:16.635962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:16.647354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 54/104] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:16.648550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8609a-42f2-4af0-86cd-28a242945e67 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:16.656170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:16.667859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc4c1f-b3e2-45ea-9b82-acd833454e86 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:16.712007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f76ba-3bb5-4557-a2d4-e17236a1ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:16.712809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f76ba-3bb5-4557-a2d4-e17236a1ec29 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.713299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f76ba-3bb5-4557-a2d4-e17236a1ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.713784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f76ba-3bb5-4557-a2d4-e17236a1ec29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.715774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f76ba-3bb5-4557-a2d4-e17236a1ec29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:16.717410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f76ba-3bb5-4557-a2d4-e17236a1ec29 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:16.719264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 51/105] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:16.720212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f76ba-3bb5-4557-a2d4-e17236a1ec29 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:16.778623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8609a-42f2-4af0-86cd-28a242945e67 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:16.814789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d6694-8d92-4ec5-9de7-8b9ad3535eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:16.815908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d6694-8d92-4ec5-9de7-8b9ad3535eed None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.816372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d6694-8d92-4ec5-9de7-8b9ad3535eed None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.816827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d6694-8d92-4ec5-9de7-8b9ad3535eed None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.827887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d6694-8d92-4ec5-9de7-8b9ad3535eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-057d6694-8d92-4ec5-9de7-8b9ad3535eed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:16.836148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:16.836981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:16.837752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:16.841061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 55/106] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:16.842012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f76ba-3bb5-4557-a2d4-e17236a1ec29 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:16.842891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917267b-d398-4993-92f9-09d1f9515661 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.843450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917267b-d398-4993-92f9-09d1f9515661 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.843966 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917267b-d398-4993-92f9-09d1f9515661 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.845856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 56/107] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:16.850792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917267b-d398-4993-92f9-09d1f9515661 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:16.879110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94268df8-63d3-496e-89e5-a12fe3618f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:16.879788 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94268df8-63d3-496e-89e5-a12fe3618f4b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:16.880081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94268df8-63d3-496e-89e5-a12fe3618f4b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:16.880421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94268df8-63d3-496e-89e5-a12fe3618f4b None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:16.881115 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94268df8-63d3-496e-89e5-a12fe3618f4b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:16.881429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94268df8-63d3-496e-89e5-a12fe3618f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:16.882833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94268df8-63d3-496e-89e5-a12fe3618f4b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:16.892929 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 52/108] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:16.893636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94268df8-63d3-496e-89e5-a12fe3618f4b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:16.904921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:16.951123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:16.967793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917267b-d398-4993-92f9-09d1f9515661 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.001703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94268df8-63d3-496e-89e5-a12fe3618f4b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.005955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d01dd-3112-4a9a-9172-0ddfce87db38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:17.006903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d01dd-3112-4a9a-9172-0ddfce87db38 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.007389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d01dd-3112-4a9a-9172-0ddfce87db38 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.008020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d01dd-3112-4a9a-9172-0ddfce87db38 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.008981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d01dd-3112-4a9a-9172-0ddfce87db38 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:17.010770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d01dd-3112-4a9a-9172-0ddfce87db38 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:17.042130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cdd00-f986-45f8-a0f7-de201b7b144a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:17.042896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cdd00-f986-45f8-a0f7-de201b7b144a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.043239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cdd00-f986-45f8-a0f7-de201b7b144a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.043526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cdd00-f986-45f8-a0f7-de201b7b144a None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.050763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349cdd00-f986-45f8-a0f7-de201b7b144a None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:17.050763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349cdd00-f986-45f8-a0f7-de201b7b144a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:17.050763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349cdd00-f986-45f8-a0f7-de201b7b144a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:17.051392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 57/109] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:36:16 2022] POST /identity/v3/projects => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:17.053274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7915d40-dee2-4add-a99a-b3f26f798211 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.053763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7915d40-dee2-4add-a99a-b3f26f798211 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.054236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7915d40-dee2-4add-a99a-b3f26f798211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.057601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 53/110] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:17.057601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cdd00-f986-45f8-a0f7-de201b7b144a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:17.087901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-f7915d40-dee2-4add-a99a-b3f26f798211 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:17.090466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f7915d40-dee2-4add-a99a-b3f26f798211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:17.140798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:17.174506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 58/111] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:17.183072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cdd00-f986-45f8-a0f7-de201b7b144a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.185044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7915d40-dee2-4add-a99a-b3f26f798211 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:17.223852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7217c-50c9-4064-8801-c3048904a1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:17.224811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7217c-50c9-4064-8801-c3048904a1d2 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.225098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7217c-50c9-4064-8801-c3048904a1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.225380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7217c-50c9-4064-8801-c3048904a1d2 None admin] PATH_INFO: `/v3/users/338aa6519e7a497bb04fd0530823cf49` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.249610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7217c-50c9-4064-8801-c3048904a1d2 None admin] RBAC: Authorizing `identity:get_user(user_id=338aa6519e7a497bb04fd0530823cf49)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:17.251110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7217c-50c9-4064-8801-c3048904a1d2 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:17.252852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 54/112] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:17 2022] GET /identity/v3/users/338aa6519e7a497bb04fd0530823cf49 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:17.258836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61664fc-a49f-4908-bf7a-38a821c105ac None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.259301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61664fc-a49f-4908-bf7a-38a821c105ac None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.259606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61664fc-a49f-4908-bf7a-38a821c105ac None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.262282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 55/113] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:17.262557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61664fc-a49f-4908-bf7a-38a821c105ac None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:17.267342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:17.340565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7915d40-dee2-4add-a99a-b3f26f798211 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.379321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660490a3-bc99-4c61-8442-146623447731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:17.381137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660490a3-bc99-4c61-8442-146623447731 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.381572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660490a3-bc99-4c61-8442-146623447731 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.382240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660490a3-bc99-4c61-8442-146623447731 None admin] PATH_INFO: `/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.384407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660490a3-bc99-4c61-8442-146623447731 None admin] RBAC: Authorizing `identity:get_user(user_id=4042a3d4d3d8483b9f9a6f1c750f1360)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:17.387568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660490a3-bc99-4c61-8442-146623447731 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:17.389376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 59/114] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:17 2022] GET /identity/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:17.390273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660490a3-bc99-4c61-8442-146623447731 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:17.394615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61664fc-a49f-4908-bf7a-38a821c105ac None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.424964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:17.436640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b998c4-37ec-4b92-a632-4f913c06d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:17.437530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b998c4-37ec-4b92-a632-4f913c06d96d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.437822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b998c4-37ec-4b92-a632-4f913c06d96d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.438107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b998c4-37ec-4b92-a632-4f913c06d96d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.452488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b998c4-37ec-4b92-a632-4f913c06d96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-59b998c4-37ec-4b92-a632-4f913c06d96d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:17.460640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:17.461475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 56/115] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:17.467931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d879b-1d69-46bf-8e4f-7feab2118e4c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.468596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d879b-1d69-46bf-8e4f-7feab2118e4c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.468596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d879b-1d69-46bf-8e4f-7feab2118e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.481174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-757d879b-1d69-46bf-8e4f-7feab2118e4c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:17.482360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-757d879b-1d69-46bf-8e4f-7feab2118e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:17.509286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660490a3-bc99-4c61-8442-146623447731 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.526475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:17.549483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5a23-1356-4c0a-8f27-ed2b1369d7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:17.550131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5a23-1356-4c0a-8f27-ed2b1369d7e8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.550415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5a23-1356-4c0a-8f27-ed2b1369d7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.550694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5a23-1356-4c0a-8f27-ed2b1369d7e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.552443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 57/116] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:17.553592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e8218-87f7-4f6e-a83b-a43b055d93ba None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.553891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e8218-87f7-4f6e-a83b-a43b055d93ba None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.554177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e8218-87f7-4f6e-a83b-a43b055d93ba None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.566773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 58/117] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:17.566773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e8218-87f7-4f6e-a83b-a43b055d93ba None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:17.633326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba5a23-1356-4c0a-8f27-ed2b1369d7e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:17.634832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba5a23-1356-4c0a-8f27-ed2b1369d7e8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:17.667017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e8218-87f7-4f6e-a83b-a43b055d93ba None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.686094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:17.705497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152921f-a97b-48e8-b1b2-cb15fc3d4f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:17.706493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152921f-a97b-48e8-b1b2-cb15fc3d4f8f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.706823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152921f-a97b-48e8-b1b2-cb15fc3d4f8f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.707110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152921f-a97b-48e8-b1b2-cb15fc3d4f8f None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.708406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152921f-a97b-48e8-b1b2-cb15fc3d4f8f None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:17.710014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152921f-a97b-48e8-b1b2-cb15fc3d4f8f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:17.711985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 59/118] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:17 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:17.712582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152921f-a97b-48e8-b1b2-cb15fc3d4f8f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:17.748652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 60/119] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:17.754432 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a2786-825b-4cbb-9132-055882f88bbc None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.754751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a2786-825b-4cbb-9132-055882f88bbc None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.755035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a2786-825b-4cbb-9132-055882f88bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.771188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-479a2786-825b-4cbb-9132-055882f88bbc None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:17.772366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-479a2786-825b-4cbb-9132-055882f88bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:17.779961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:17.820009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152921f-a97b-48e8-b1b2-cb15fc3d4f8f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.840602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 61/120] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:17.841451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a2786-825b-4cbb-9132-055882f88bbc None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:17.863425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962f79b-a385-480d-957b-551c667e7e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:17.864644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962f79b-a385-480d-957b-551c667e7e57 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.864927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962f79b-a385-480d-957b-551c667e7e57 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.865353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962f79b-a385-480d-957b-551c667e7e57 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:17.866145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962f79b-a385-480d-957b-551c667e7e57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:17.866575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962f79b-a385-480d-957b-551c667e7e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:17.868045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962f79b-a385-480d-957b-551c667e7e57 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:17.889611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 60/121] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:17.889611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962f79b-a385-480d-957b-551c667e7e57 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:17.941234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:17.958989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a2786-825b-4cbb-9132-055882f88bbc None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.996726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb0eb1-cedb-49af-a63c-cb5f80631ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:17.997055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962f79b-a385-480d-957b-551c667e7e57 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:17.997596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb0eb1-cedb-49af-a63c-cb5f80631ab2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:17.997881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb0eb1-cedb-49af-a63c-cb5f80631ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:17.998152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb0eb1-cedb-49af-a63c-cb5f80631ab2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.008209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb0eb1-cedb-49af-a63c-cb5f80631ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-2ccb0eb1-cedb-49af-a63c-cb5f80631ab2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:18.016653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:18.017523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:18.018305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 62/122] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:18.018992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb0eb1-cedb-49af-a63c-cb5f80631ab2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:18.028375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:18.041002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f282ba7-1927-413d-b023-ab3cbb218218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:18.042637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f282ba7-1927-413d-b023-ab3cbb218218 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:18.043342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f282ba7-1927-413d-b023-ab3cbb218218 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:18.043770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f282ba7-1927-413d-b023-ab3cbb218218 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f/users/4042a3d4d3d8483b9f9a6f1c750f1360/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.046831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f282ba7-1927-413d-b023-ab3cbb218218 None admin] RBAC: Authorizing `identity:create_grant(project_id=54d37e7eb5474cf387beec6e56303b4f, user_id=4042a3d4d3d8483b9f9a6f1c750f1360, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:18.049166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f282ba7-1927-413d-b023-ab3cbb218218 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:18.070757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 61/123] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:36:17 2022] PUT /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f/users/4042a3d4d3d8483b9f9a6f1c750f1360/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:18.070757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f282ba7-1927-413d-b023-ab3cbb218218 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:18.144932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb0eb1-cedb-49af-a63c-cb5f80631ab2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:18.197520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01b197-b2f1-4df9-854f-5b4e673e4485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:18.198544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01b197-b2f1-4df9-854f-5b4e673e4485 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:18.199514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01b197-b2f1-4df9-854f-5b4e673e4485 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:18.199903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01b197-b2f1-4df9-854f-5b4e673e4485 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.201349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01b197-b2f1-4df9-854f-5b4e673e4485 None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:18.203259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01b197-b2f1-4df9-854f-5b4e673e4485 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:18.205620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 63/124] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:18 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:18.206763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01b197-b2f1-4df9-854f-5b4e673e4485 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:18.218350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f282ba7-1927-413d-b023-ab3cbb218218 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:18.231128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:18.265841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c399e-8291-46d9-aeec-7fbf36a40f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:18.266680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c399e-8291-46d9-aeec-7fbf36a40f40 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:18.267015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c399e-8291-46d9-aeec-7fbf36a40f40 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:18.267292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c399e-8291-46d9-aeec-7fbf36a40f40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.281663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c399e-8291-46d9-aeec-7fbf36a40f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-081c399e-8291-46d9-aeec-7fbf36a40f40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:18.290104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:18.291008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:18.291784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:18.293352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 62/125] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:18.294113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c399e-8291-46d9-aeec-7fbf36a40f40 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:18.296243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:18.346032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01b197-b2f1-4df9-854f-5b4e673e4485 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:18.393724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a907d5d-eca7-4133-830f-8833fc9cd993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:18.394610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a907d5d-eca7-4133-830f-8833fc9cd993 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:18.395068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a907d5d-eca7-4133-830f-8833fc9cd993 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:18.395365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a907d5d-eca7-4133-830f-8833fc9cd993 None admin] PATH_INFO: `/v3/users/38729f2fb8864984b35336009a142811` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.397543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a907d5d-eca7-4133-830f-8833fc9cd993 None admin] RBAC: Authorizing `identity:get_user(user_id=38729f2fb8864984b35336009a142811)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:18.399567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a907d5d-eca7-4133-830f-8833fc9cd993 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:18.401575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 64/126] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:18 2022] GET /identity/v3/users/38729f2fb8864984b35336009a142811 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:18.406874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a907d5d-eca7-4133-830f-8833fc9cd993 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:18.412633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c399e-8291-46d9-aeec-7fbf36a40f40 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:18.458403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc309a-d385-4604-83d9-07321c40ab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:18.459330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc309a-d385-4604-83d9-07321c40ab82 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:18.459754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc309a-d385-4604-83d9-07321c40ab82 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:18.460168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc309a-d385-4604-83d9-07321c40ab82 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.461112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc309a-d385-4604-83d9-07321c40ab82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:18.461600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc309a-d385-4604-83d9-07321c40ab82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:18.463287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc309a-d385-4604-83d9-07321c40ab82 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:18.477655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 63/127] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:18.478973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc309a-d385-4604-83d9-07321c40ab82 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:18.521402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:18.530943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a907d5d-eca7-4133-830f-8833fc9cd993 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:18.567265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f7f49-e097-4969-a41e-625c45e59114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:18.568363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f7f49-e097-4969-a41e-625c45e59114 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:18.568841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f7f49-e097-4969-a41e-625c45e59114 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:18.569428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f7f49-e097-4969-a41e-625c45e59114 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.570446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f7f49-e097-4969-a41e-625c45e59114 None admin] RBAC: Adding query filter params (role.id=aab9a25c2ea047f89513ed57a120fa57, scope.project.id=54d37e7eb5474cf387beec6e56303b4f, user.id=338aa6519e7a497bb04fd0530823cf49) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:18.570978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f7f49-e097-4969-a41e-625c45e59114 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:18.572918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f7f49-e097-4969-a41e-625c45e59114 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:18.578896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 65/128] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:18 2022] GET /identity/v3/role_assignments?user.id=338aa6519e7a497bb04fd0530823cf49&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=aab9a25c2ea047f89513ed57a120fa57 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:18.580075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f7f49-e097-4969-a41e-625c45e59114 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:18.587387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:18.646706 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc309a-d385-4604-83d9-07321c40ab82 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:18.703904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd044f-dc1d-4021-8cdf-e05fbb8f91ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:18.705068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd044f-dc1d-4021-8cdf-e05fbb8f91ce None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:18.705616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd044f-dc1d-4021-8cdf-e05fbb8f91ce None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:18.706091 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd044f-dc1d-4021-8cdf-e05fbb8f91ce None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.707162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd044f-dc1d-4021-8cdf-e05fbb8f91ce None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:18.707698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd044f-dc1d-4021-8cdf-e05fbb8f91ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:18.709302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd044f-dc1d-4021-8cdf-e05fbb8f91ce None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:18.722961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 64/129] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:18.724222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd044f-dc1d-4021-8cdf-e05fbb8f91ce None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:18.738921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f7f49-e097-4969-a41e-625c45e59114 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:18.806124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003860f3-688f-4839-82d0-251ee6d091be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:18.809187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003860f3-688f-4839-82d0-251ee6d091be None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:18.810840 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003860f3-688f-4839-82d0-251ee6d091be None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:18.811171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003860f3-688f-4839-82d0-251ee6d091be None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.813465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003860f3-688f-4839-82d0-251ee6d091be None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:18.817702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003860f3-688f-4839-82d0-251ee6d091be None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:18.824601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 66/130] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:18 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:18.825458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003860f3-688f-4839-82d0-251ee6d091be None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:18.863627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:18.874549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd044f-dc1d-4021-8cdf-e05fbb8f91ce None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:18.915912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694b34e-1cd9-404f-969d-d1c65a603662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:18.917092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694b34e-1cd9-404f-969d-d1c65a603662 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:18.917903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694b34e-1cd9-404f-969d-d1c65a603662 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:18.918372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694b34e-1cd9-404f-969d-d1c65a603662 None admin] PATH_INFO: `/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:18.920155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694b34e-1cd9-404f-969d-d1c65a603662 None admin] RBAC: Authorizing `identity:get_user(user_id=4042a3d4d3d8483b9f9a6f1c750f1360)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:18.921615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694b34e-1cd9-404f-969d-d1c65a603662 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:18.923505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 65/131] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:18 2022] GET /identity/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:18.924230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694b34e-1cd9-404f-969d-d1c65a603662 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:19.006121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:19.040249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694b34e-1cd9-404f-969d-d1c65a603662 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:19.052277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003860f3-688f-4839-82d0-251ee6d091be None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:19.096153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d68c66-7564-48d3-868d-23240d5d1bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:19.097037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d68c66-7564-48d3-868d-23240d5d1bb9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:19.097324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d68c66-7564-48d3-868d-23240d5d1bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:19.097605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d68c66-7564-48d3-868d-23240d5d1bb9 None admin] PATH_INFO: `/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:19.098845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d68c66-7564-48d3-868d-23240d5d1bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=4042a3d4d3d8483b9f9a6f1c750f1360)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:19.100324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d68c66-7564-48d3-868d-23240d5d1bb9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:19.101954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 67/132] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:18 2022] GET /identity/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360 => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:19.102718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d68c66-7564-48d3-868d-23240d5d1bb9 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:19.117654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e97165-1365-4113-a40f-c44db7e739a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:19.118563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e97165-1365-4113-a40f-c44db7e739a3 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:19.118880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e97165-1365-4113-a40f-c44db7e739a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:19.119162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e97165-1365-4113-a40f-c44db7e739a3 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f/users/38729f2fb8864984b35336009a142811/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:19.121944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e97165-1365-4113-a40f-c44db7e739a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=54d37e7eb5474cf387beec6e56303b4f, user_id=38729f2fb8864984b35336009a142811, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:19.123428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e97165-1365-4113-a40f-c44db7e739a3 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:19.147475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 66/133] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:36:18 2022] PUT /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f/users/38729f2fb8864984b35336009a142811/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:19.148679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e97165-1365-4113-a40f-c44db7e739a3 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:19.192433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:19.256120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d68c66-7564-48d3-868d-23240d5d1bb9 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:19.295569 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79ee75-b6fe-4940-a627-e1aee6efd60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:19.296363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79ee75-b6fe-4940-a627-e1aee6efd60e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:19.296649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79ee75-b6fe-4940-a627-e1aee6efd60e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:19.297031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79ee75-b6fe-4940-a627-e1aee6efd60e None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:19.298299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79ee75-b6fe-4940-a627-e1aee6efd60e None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:19.300024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79ee75-b6fe-4940-a627-e1aee6efd60e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:19.301890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 68/134] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:19 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:19.307485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79ee75-b6fe-4940-a627-e1aee6efd60e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:19.324386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:19.334213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e97165-1365-4113-a40f-c44db7e739a3 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:19.396077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d3cee-0dbb-4d8b-b0b6-3d6818500d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:19.397071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d3cee-0dbb-4d8b-b0b6-3d6818500d24 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:19.397366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d3cee-0dbb-4d8b-b0b6-3d6818500d24 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:19.397650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d3cee-0dbb-4d8b-b0b6-3d6818500d24 None admin] PATH_INFO: `/v3/projects/0e9d0a88b7cf41aa8d2ad10351114364` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:19.399244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d3cee-0dbb-4d8b-b0b6-3d6818500d24 None admin] RBAC: Authorizing `identity:get_project(project_id=0e9d0a88b7cf41aa8d2ad10351114364)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:19.400841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d3cee-0dbb-4d8b-b0b6-3d6818500d24 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:19.403531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 67/135] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:19 2022] GET /identity/v3/projects/0e9d0a88b7cf41aa8d2ad10351114364 => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:19.409308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d3cee-0dbb-4d8b-b0b6-3d6818500d24 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:19.420670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79ee75-b6fe-4940-a627-e1aee6efd60e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:19.458656 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c855db-65a5-4c54-82b7-eb7b0261d567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:19.459599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c855db-65a5-4c54-82b7-eb7b0261d567 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:19.459992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c855db-65a5-4c54-82b7-eb7b0261d567 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:19.460280 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c855db-65a5-4c54-82b7-eb7b0261d567 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f/users/4042a3d4d3d8483b9f9a6f1c750f1360/roles/7fb980f85eb04293a0d1dba515bf0a2c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:19.462889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c855db-65a5-4c54-82b7-eb7b0261d567 None admin] RBAC: Authorizing `identity:create_grant(project_id=54d37e7eb5474cf387beec6e56303b4f, user_id=4042a3d4d3d8483b9f9a6f1c750f1360, role_id=7fb980f85eb04293a0d1dba515bf0a2c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:19.465261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c855db-65a5-4c54-82b7-eb7b0261d567 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:19.482984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 69/136] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:36:19 2022] PUT /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f/users/4042a3d4d3d8483b9f9a6f1c750f1360/roles/7fb980f85eb04293a0d1dba515bf0a2c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:36:19.583543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:19.597686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d3cee-0dbb-4d8b-b0b6-3d6818500d24 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:19.645399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5f4aa-97c0-4db1-9a75-10ff5bc36b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:19.646670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5f4aa-97c0-4db1-9a75-10ff5bc36b9f None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:19.647188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5f4aa-97c0-4db1-9a75-10ff5bc36b9f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:19.647671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5f4aa-97c0-4db1-9a75-10ff5bc36b9f None admin] PATH_INFO: `/v3/projects/0e9d0a88b7cf41aa8d2ad10351114364/users/4042a3d4d3d8483b9f9a6f1c750f1360/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:19.656414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5f4aa-97c0-4db1-9a75-10ff5bc36b9f None admin] RBAC: Authorizing `identity:create_grant(project_id=0e9d0a88b7cf41aa8d2ad10351114364, user_id=4042a3d4d3d8483b9f9a6f1c750f1360, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:19.658208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5f4aa-97c0-4db1-9a75-10ff5bc36b9f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:19.681557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f055b-4745-408f-b354-144deff99ea7 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:19.682187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f055b-4745-408f-b354-144deff99ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:19.682686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f055b-4745-408f-b354-144deff99ea7 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:19.684933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 70/137] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:36:19.694385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 68/138] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:36:19 2022] PUT /identity/v3/projects/0e9d0a88b7cf41aa8d2ad10351114364/users/4042a3d4d3d8483b9f9a6f1c750f1360/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:19.710493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45c9d5-05cc-4296-8e71-7b40e54453cb None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:19.711130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45c9d5-05cc-4296-8e71-7b40e54453cb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:19.711628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45c9d5-05cc-4296-8e71-7b40e54453cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:19.736791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-8c45c9d5-05cc-4296-8e71-7b40e54453cb None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:19.738289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8c45c9d5-05cc-4296-8e71-7b40e54453cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:19.805839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:19.815529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 71/139] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:19.864340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5f4aa-97c0-4db1-9a75-10ff5bc36b9f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:20.009714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5f4aa-97c0-4db1-9a75-10ff5bc36b9f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:20.058751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7526d-a56a-445a-aa78-d38e92d5305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:20.059965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7526d-a56a-445a-aa78-d38e92d5305e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:20.060461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7526d-a56a-445a-aa78-d38e92d5305e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:20.060956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7526d-a56a-445a-aa78-d38e92d5305e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:20.063632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7526d-a56a-445a-aa78-d38e92d5305e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:20.065143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7526d-a56a-445a-aa78-d38e92d5305e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:20.066917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 69/140] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:20.072993 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45c9d5-05cc-4296-8e71-7b40e54453cb None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:20.251830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45c9d5-05cc-4296-8e71-7b40e54453cb None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:20.290885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79503385-2c82-4c90-b676-4a342ca09c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:20.291645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79503385-2c82-4c90-b676-4a342ca09c01 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:20.291853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79503385-2c82-4c90-b676-4a342ca09c01 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:20.292132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79503385-2c82-4c90-b676-4a342ca09c01 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:20.292884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79503385-2c82-4c90-b676-4a342ca09c01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:20.294342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79503385-2c82-4c90-b676-4a342ca09c01 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:20.341338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 72/141] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:36:20 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:20.763430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97357314-f658-4aee-962a-a3b606617b94 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:20.763945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97357314-f658-4aee-962a-a3b606617b94 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:20.764036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97357314-f658-4aee-962a-a3b606617b94 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:20.765635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 70/142] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:20.771045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e43f9-2984-4c11-91bb-3fb6a998edf4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:20.771343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e43f9-2984-4c11-91bb-3fb6a998edf4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:20.771628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e43f9-2984-4c11-91bb-3fb6a998edf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:20.790485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-503e43f9-2984-4c11-91bb-3fb6a998edf4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:20.791734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-503e43f9-2984-4c11-91bb-3fb6a998edf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:20.856663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:20.885911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 73/143] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:20.921734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97357314-f658-4aee-962a-a3b606617b94 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:20.953109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:21.074713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97357314-f658-4aee-962a-a3b606617b94 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:21.114973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f69614-d808-4f80-a34c-7b5b1a712cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:21.115789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f69614-d808-4f80-a34c-7b5b1a712cf3 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:21.116079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f69614-d808-4f80-a34c-7b5b1a712cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:21.116365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f69614-d808-4f80-a34c-7b5b1a712cf3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:21.131798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f69614-d808-4f80-a34c-7b5b1a712cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-98f69614-d808-4f80-a34c-7b5b1a712cf3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:21.140072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:21.140962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:21.142692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 71/144] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:21.147049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e43f9-2984-4c11-91bb-3fb6a998edf4 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:21.295692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e43f9-2984-4c11-91bb-3fb6a998edf4 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:21.337583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901bd68-da3f-4f5b-98eb-919ce20bed72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:21.338181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901bd68-da3f-4f5b-98eb-919ce20bed72 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:21.338468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901bd68-da3f-4f5b-98eb-919ce20bed72 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:21.338767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901bd68-da3f-4f5b-98eb-919ce20bed72 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:21.340246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901bd68-da3f-4f5b-98eb-919ce20bed72 None admin] RBAC: Adding query filter params (name=member) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:21.340562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901bd68-da3f-4f5b-98eb-919ce20bed72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:21.341981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901bd68-da3f-4f5b-98eb-919ce20bed72 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:21.354580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 74/145] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:21.360668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f69614-d808-4f80-a34c-7b5b1a712cf3 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:21.374779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c58de-4e0a-4f75-9d3d-30551ac1b77b None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:21.375075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c58de-4e0a-4f75-9d3d-30551ac1b77b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:21.375345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c58de-4e0a-4f75-9d3d-30551ac1b77b None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:21.377025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 75/146] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:21.387943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47e92a-6ec3-44d2-8e22-fcfb6ba2b74c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:21.388218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47e92a-6ec3-44d2-8e22-fcfb6ba2b74c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:21.388499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47e92a-6ec3-44d2-8e22-fcfb6ba2b74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:21.418290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9a47e92a-6ec3-44d2-8e22-fcfb6ba2b74c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:21.419642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9a47e92a-6ec3-44d2-8e22-fcfb6ba2b74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:21.458468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:21.485883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f69614-d808-4f80-a34c-7b5b1a712cf3 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:21.503892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:21.525067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fdc24-e58d-4233-ad63-e7b7476217c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:21.526106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fdc24-e58d-4233-ad63-e7b7476217c5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:21.526393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fdc24-e58d-4233-ad63-e7b7476217c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:21.527228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fdc24-e58d-4233-ad63-e7b7476217c5 None admin] PATH_INFO: `/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:21.528484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fdc24-e58d-4233-ad63-e7b7476217c5 None admin] RBAC: Authorizing `identity:get_user(user_id=4042a3d4d3d8483b9f9a6f1c750f1360)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:21.530291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fdc24-e58d-4233-ad63-e7b7476217c5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:21.532669 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 72/147] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:21 2022] GET /identity/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:21.538358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fdc24-e58d-4233-ad63-e7b7476217c5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:21.564563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 76/148] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:21.655393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47e92a-6ec3-44d2-8e22-fcfb6ba2b74c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:21.667292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fdc24-e58d-4233-ad63-e7b7476217c5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:21.711699 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c45db9-b538-49fb-aa27-3eb2e4b8ace5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:21.712675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c45db9-b538-49fb-aa27-3eb2e4b8ace5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:21.712964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c45db9-b538-49fb-aa27-3eb2e4b8ace5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:21.713244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c45db9-b538-49fb-aa27-3eb2e4b8ace5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:21.731000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:21.810695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c45db9-b538-49fb-aa27-3eb2e4b8ace5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:21.812202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c45db9-b538-49fb-aa27-3eb2e4b8ace5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:21.867257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47e92a-6ec3-44d2-8e22-fcfb6ba2b74c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:21.927843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fcdc2-52fd-4c91-a5ea-3bc2707c04a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:21.928584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fcdc2-52fd-4c91-a5ea-3bc2707c04a3 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:21.928875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fcdc2-52fd-4c91-a5ea-3bc2707c04a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:21.929161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fcdc2-52fd-4c91-a5ea-3bc2707c04a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:21.937716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 73/149] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 15:36:21.943697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c45db9-b538-49fb-aa27-3eb2e4b8ace5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:21.953259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179fcdc2-52fd-4c91-a5ea-3bc2707c04a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-179fcdc2-52fd-4c91-a5ea-3bc2707c04a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:21.966124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:21.968701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:21.970639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 77/150] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:21.978780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fcdc2-52fd-4c91-a5ea-3bc2707c04a3 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:21.999458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:22.045007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:22.069098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c45db9-b538-49fb-aa27-3eb2e4b8ace5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:22.110523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d5157-e061-41c4-bea9-731e3ded32fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:22.111843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d5157-e061-41c4-bea9-731e3ded32fa None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:22.112294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d5157-e061-41c4-bea9-731e3ded32fa None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:22.113510 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d5157-e061-41c4-bea9-731e3ded32fa None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:22.114947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d5157-e061-41c4-bea9-731e3ded32fa None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:22.116552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d5157-e061-41c4-bea9-731e3ded32fa None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:22.118891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 74/151] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:21 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:22.125723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d5157-e061-41c4-bea9-731e3ded32fa None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:22.165629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fcdc2-52fd-4c91-a5ea-3bc2707c04a3 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:22.223099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29222084-7556-4cd4-93b8-9bf3b910f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:22.223798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29222084-7556-4cd4-93b8-9bf3b910f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:22.224259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29222084-7556-4cd4-93b8-9bf3b910f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:22.224718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29222084-7556-4cd4-93b8-9bf3b910f6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:22.225626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29222084-7556-4cd4-93b8-9bf3b910f6e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:22.226179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29222084-7556-4cd4-93b8-9bf3b910f6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:22.227873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29222084-7556-4cd4-93b8-9bf3b910f6e8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:22.243455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d5157-e061-41c4-bea9-731e3ded32fa None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:22.246496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 78/152] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:22.252615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29222084-7556-4cd4-93b8-9bf3b910f6e8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:22.259067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:22.301659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dc590-bfb3-4151-ae9c-d33e042092b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:22.302674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dc590-bfb3-4151-ae9c-d33e042092b5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:22.304276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dc590-bfb3-4151-ae9c-d33e042092b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:22.304876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dc590-bfb3-4151-ae9c-d33e042092b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:22.307210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10dc590-bfb3-4151-ae9c-d33e042092b5 None admin] RBAC: Adding query filter params (role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=54d37e7eb5474cf387beec6e56303b4f, user.id=4042a3d4d3d8483b9f9a6f1c750f1360) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:22.307969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10dc590-bfb3-4151-ae9c-d33e042092b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:22.310317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10dc590-bfb3-4151-ae9c-d33e042092b5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:22.330799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 75/153] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:22 2022] GET /identity/v3/role_assignments?user.id=4042a3d4d3d8483b9f9a6f1c750f1360&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:22.413500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:22.425546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29222084-7556-4cd4-93b8-9bf3b910f6e8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:22.467482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcfec9-107c-4179-83c0-c658f7f54ee3 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:22.467795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcfec9-107c-4179-83c0-c658f7f54ee3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:22.468050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcfec9-107c-4179-83c0-c658f7f54ee3 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:22.469426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 76/154] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:36:22.474956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a8230-1450-4604-b5cd-185cb9c8b2e1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:22.475192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a8230-1450-4604-b5cd-185cb9c8b2e1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:22.475453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a8230-1450-4604-b5cd-185cb9c8b2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:22.485074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29aeb0-f138-4128-a6df-1bb79d392be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:22.486324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29aeb0-f138-4128-a6df-1bb79d392be5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:22.486839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29aeb0-f138-4128-a6df-1bb79d392be5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:22.487403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29aeb0-f138-4128-a6df-1bb79d392be5 None admin] PATH_INFO: `/v3/users/338aa6519e7a497bb04fd0530823cf49` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:22.491499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29aeb0-f138-4128-a6df-1bb79d392be5 None admin] RBAC: Authorizing `identity:get_user(user_id=338aa6519e7a497bb04fd0530823cf49)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:22.493279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29aeb0-f138-4128-a6df-1bb79d392be5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:22.495372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 79/155] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:22 2022] GET /identity/v3/users/338aa6519e7a497bb04fd0530823cf49 => generated 289 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:22.502900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-839a8230-1450-4604-b5cd-185cb9c8b2e1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:22.504168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-839a8230-1450-4604-b5cd-185cb9c8b2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:22.509499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29aeb0-f138-4128-a6df-1bb79d392be5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:22.677193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 77/156] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:22.717684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29aeb0-f138-4128-a6df-1bb79d392be5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:22.757566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a8230-1450-4604-b5cd-185cb9c8b2e1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:22.761480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd415b8-e247-4204-8b32-4dbf7f1678fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:22.766892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd415b8-e247-4204-8b32-4dbf7f1678fc None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:22.767179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd415b8-e247-4204-8b32-4dbf7f1678fc None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:22.767460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd415b8-e247-4204-8b32-4dbf7f1678fc None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:22.770416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd415b8-e247-4204-8b32-4dbf7f1678fc None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:22.771947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd415b8-e247-4204-8b32-4dbf7f1678fc None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:22.773973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 80/157] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:22 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:22.788112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd415b8-e247-4204-8b32-4dbf7f1678fc None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:22.865210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:22.947366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a8230-1450-4604-b5cd-185cb9c8b2e1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:22.975832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:22.999247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4e4c8-4eb9-4b09-ba54-074ab01706c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:23.000056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4e4c8-4eb9-4b09-ba54-074ab01706c5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:23.000343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4e4c8-4eb9-4b09-ba54-074ab01706c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:23.000623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4e4c8-4eb9-4b09-ba54-074ab01706c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:23.017790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4e4c8-4eb9-4b09-ba54-074ab01706c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-e0f4e4c8-4eb9-4b09-ba54-074ab01706c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:23.025732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:23.027235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:23.028005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:23.039151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd415b8-e247-4204-8b32-4dbf7f1678fc None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:23.043379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 78/158] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:23.051888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4e4c8-4eb9-4b09-ba54-074ab01706c5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:23.085018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0c188-f8e3-49ae-b0d0-b050ca2000ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:23.085923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0c188-f8e3-49ae-b0d0-b050ca2000ba None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:23.086207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0c188-f8e3-49ae-b0d0-b050ca2000ba None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:23.086484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0c188-f8e3-49ae-b0d0-b050ca2000ba None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f/users/338aa6519e7a497bb04fd0530823cf49/roles/aab9a25c2ea047f89513ed57a120fa57` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:23.088689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0c188-f8e3-49ae-b0d0-b050ca2000ba None admin] RBAC: Authorizing `identity:create_grant(project_id=54d37e7eb5474cf387beec6e56303b4f, user_id=338aa6519e7a497bb04fd0530823cf49, role_id=aab9a25c2ea047f89513ed57a120fa57)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:23.090685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0c188-f8e3-49ae-b0d0-b050ca2000ba None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:23.123875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 81/159] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:36:22 2022] PUT /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f/users/338aa6519e7a497bb04fd0530823cf49/roles/aab9a25c2ea047f89513ed57a120fa57 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:36:23.259611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4e4c8-4eb9-4b09-ba54-074ab01706c5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:23.302428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d907c87-9020-45fe-a731-f3eaae321ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:23.303066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d907c87-9020-45fe-a731-f3eaae321ecf None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:23.303352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d907c87-9020-45fe-a731-f3eaae321ecf None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:23.303633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d907c87-9020-45fe-a731-f3eaae321ecf None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:23.304419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d907c87-9020-45fe-a731-f3eaae321ecf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:23.304733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d907c87-9020-45fe-a731-f3eaae321ecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:23.306206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d907c87-9020-45fe-a731-f3eaae321ecf None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:23.319044 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 79/160] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:23.325390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0c188-f8e3-49ae-b0d0-b050ca2000ba None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:23.459162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:23.468232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0c188-f8e3-49ae-b0d0-b050ca2000ba None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:23.535002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ab523-5c80-4d24-aef7-d21564c0b093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:23.535905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ab523-5c80-4d24-aef7-d21564c0b093 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:23.536194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ab523-5c80-4d24-aef7-d21564c0b093 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:23.536475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ab523-5c80-4d24-aef7-d21564c0b093 None admin] PATH_INFO: `/v3/users/38729f2fb8864984b35336009a142811` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:23.539186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ab523-5c80-4d24-aef7-d21564c0b093 None admin] RBAC: Authorizing `identity:get_user(user_id=38729f2fb8864984b35336009a142811)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:23.540715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ab523-5c80-4d24-aef7-d21564c0b093 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:23.543329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 82/161] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:23 2022] GET /identity/v3/users/38729f2fb8864984b35336009a142811 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:23.548707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d907c87-9020-45fe-a731-f3eaae321ecf None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:23.588529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:23.668480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d907c87-9020-45fe-a731-f3eaae321ecf None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:23.706494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96572a-0584-4cc9-bf24-7ff40623a70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:23.707181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96572a-0584-4cc9-bf24-7ff40623a70d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:23.707468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96572a-0584-4cc9-bf24-7ff40623a70d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:23.707747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96572a-0584-4cc9-bf24-7ff40623a70d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:23.788961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda4fb8-d04c-4df6-9e9a-5cabf85bedbd None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:23.789263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda4fb8-d04c-4df6-9e9a-5cabf85bedbd None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:23.789527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda4fb8-d04c-4df6-9e9a-5cabf85bedbd None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:23.790949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 83/162] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:23.801339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96572a-0584-4cc9-bf24-7ff40623a70d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:23.803302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d771b-0ac1-44dd-9796-07f218a3d6b5 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:23.803593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d771b-0ac1-44dd-9796-07f218a3d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:23.803873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d771b-0ac1-44dd-9796-07f218a3d6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:23.810177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96572a-0584-4cc9-bf24-7ff40623a70d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:23.826964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-d39d771b-0ac1-44dd-9796-07f218a3d6b5 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:23.828357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d39d771b-0ac1-44dd-9796-07f218a3d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:23.850440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:23.947973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 80/163] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:23.956916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96572a-0584-4cc9-bf24-7ff40623a70d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:23.973258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 84/164] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:24.013634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d771b-0ac1-44dd-9796-07f218a3d6b5 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:24.121879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:24.133139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d771b-0ac1-44dd-9796-07f218a3d6b5 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:24.150798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96572a-0584-4cc9-bf24-7ff40623a70d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:24.168911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5660cfc-cca6-45e3-a450-61ae5f4b1e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:24.170031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5660cfc-cca6-45e3-a450-61ae5f4b1e0f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:24.170512 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5660cfc-cca6-45e3-a450-61ae5f4b1e0f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:24.171013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5660cfc-cca6-45e3-a450-61ae5f4b1e0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:24.182949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5660cfc-cca6-45e3-a450-61ae5f4b1e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-b5660cfc-cca6-45e3-a450-61ae5f4b1e0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:24.191750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:24.192631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:24.193412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:24.196055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 85/165] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:24.197542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70497dc-9707-49e9-992a-a7a802c2b19f None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:24.198024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70497dc-9707-49e9-992a-a7a802c2b19f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:24.199076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70497dc-9707-49e9-992a-a7a802c2b19f None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:24.200853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 86/166] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:24.208098 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6c7d6-5594-4ac4-8f5d-985721e81e92 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:24.208626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6c7d6-5594-4ac4-8f5d-985721e81e92 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:24.209113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6c7d6-5594-4ac4-8f5d-985721e81e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:24.212068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:24.227255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-30e6c7d6-5594-4ac4-8f5d-985721e81e92 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:24.228871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-30e6c7d6-5594-4ac4-8f5d-985721e81e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:24.264010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfd26b-a726-4a94-9f96-85b984640f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:24.264897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfd26b-a726-4a94-9f96-85b984640f6b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:24.265287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfd26b-a726-4a94-9f96-85b984640f6b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:24.265460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfd26b-a726-4a94-9f96-85b984640f6b None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:24.267041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfd26b-a726-4a94-9f96-85b984640f6b None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:24.268125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfd26b-a726-4a94-9f96-85b984640f6b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:24.270834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 81/167] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:23 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:24.271622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfd26b-a726-4a94-9f96-85b984640f6b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:24.300677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 87/168] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:24.301899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6c7d6-5594-4ac4-8f5d-985721e81e92 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:24.408545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:24.430250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6c7d6-5594-4ac4-8f5d-985721e81e92 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:24.463183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfd26b-a726-4a94-9f96-85b984640f6b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:24.488819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86cb46-553b-4964-b8d5-9cb950894c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:24.489489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86cb46-553b-4964-b8d5-9cb950894c85 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:24.489778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86cb46-553b-4964-b8d5-9cb950894c85 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:24.490056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86cb46-553b-4964-b8d5-9cb950894c85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:24.490922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d86cb46-553b-4964-b8d5-9cb950894c85 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=54d37e7eb5474cf387beec6e56303b4f, user.id=38729f2fb8864984b35336009a142811) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:24.491236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d86cb46-553b-4964-b8d5-9cb950894c85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:24.495642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d86cb46-553b-4964-b8d5-9cb950894c85 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:24.511856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 88/169] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:24 2022] GET /identity/v3/role_assignments?user.id=38729f2fb8864984b35336009a142811&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=433478cc318a467c931f13a757f161fa => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:24.513045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86cb46-553b-4964-b8d5-9cb950894c85 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:24.529774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935d89c-a36d-42b2-a438-d9eaf075f8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:24.530460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935d89c-a36d-42b2-a438-d9eaf075f8aa None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:24.530766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935d89c-a36d-42b2-a438-d9eaf075f8aa None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:24.531051 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935d89c-a36d-42b2-a438-d9eaf075f8aa None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:24.531874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935d89c-a36d-42b2-a438-d9eaf075f8aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:24.532187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935d89c-a36d-42b2-a438-d9eaf075f8aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:24.533731 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935d89c-a36d-42b2-a438-d9eaf075f8aa None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:24.552240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 82/170] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:24.558534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935d89c-a36d-42b2-a438-d9eaf075f8aa None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:24.681452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:24.727497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86cb46-553b-4964-b8d5-9cb950894c85 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:24.770507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935d89c-a36d-42b2-a438-d9eaf075f8aa None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:24.799947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e0289-40c5-4e49-8900-609a28356752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:24.800752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e0289-40c5-4e49-8900-609a28356752 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:24.801034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e0289-40c5-4e49-8900-609a28356752 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:24.801307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e0289-40c5-4e49-8900-609a28356752 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:24.825525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e0289-40c5-4e49-8900-609a28356752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-932e0289-40c5-4e49-8900-609a28356752 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:24.833420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:24.834356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:24.835218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 89/171] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:24.842448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e0289-40c5-4e49-8900-609a28356752 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:24.847000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:24.862905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad0eef-3338-4577-bd6f-1f10a4678913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:24.862905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad0eef-3338-4577-bd6f-1f10a4678913 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:24.862905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad0eef-3338-4577-bd6f-1f10a4678913 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:24.862905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad0eef-3338-4577-bd6f-1f10a4678913 None admin] PATH_INFO: `/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:24.863391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad0eef-3338-4577-bd6f-1f10a4678913 None admin] RBAC: Authorizing `identity:get_user(user_id=4042a3d4d3d8483b9f9a6f1c750f1360)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:24.863391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad0eef-3338-4577-bd6f-1f10a4678913 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:24.863707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 83/172] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:24 2022] GET /identity/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360 => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:24.869092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad0eef-3338-4577-bd6f-1f10a4678913 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:25.024005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e0289-40c5-4e49-8900-609a28356752 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:25.078179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868021b-4f95-4d7b-b082-58900c53a9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:25.078890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868021b-4f95-4d7b-b082-58900c53a9d1 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.079176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868021b-4f95-4d7b-b082-58900c53a9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.079451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868021b-4f95-4d7b-b082-58900c53a9d1 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.080206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868021b-4f95-4d7b-b082-58900c53a9d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:25.080515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868021b-4f95-4d7b-b082-58900c53a9d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:25.081922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868021b-4f95-4d7b-b082-58900c53a9d1 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:25.095730 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 90/173] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:25.102426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868021b-4f95-4d7b-b082-58900c53a9d1 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:25.105564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad0eef-3338-4577-bd6f-1f10a4678913 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:25.173375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:25.200694 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d18870-d3a8-4453-ab8d-1bc712e9b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:25.201365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d18870-d3a8-4453-ab8d-1bc712e9b47f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.201650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d18870-d3a8-4453-ab8d-1bc712e9b47f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.201927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d18870-d3a8-4453-ab8d-1bc712e9b47f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.267430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:25.358606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868021b-4f95-4d7b-b082-58900c53a9d1 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:25.379977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d18870-d3a8-4453-ab8d-1bc712e9b47f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:25.381494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d18870-d3a8-4453-ab8d-1bc712e9b47f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:25.435478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21f303-412a-4039-b60d-20aafcf42d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:25.436378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21f303-412a-4039-b60d-20aafcf42d57 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.436657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21f303-412a-4039-b60d-20aafcf42d57 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.436930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21f303-412a-4039-b60d-20aafcf42d57 None admin] PATH_INFO: `/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.438200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21f303-412a-4039-b60d-20aafcf42d57 None admin] RBAC: Authorizing `identity:get_user(user_id=4042a3d4d3d8483b9f9a6f1c750f1360)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:25.439705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21f303-412a-4039-b60d-20aafcf42d57 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:25.441343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 91/174] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:25 2022] GET /identity/v3/users/4042a3d4d3d8483b9f9a6f1c750f1360 => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:25.448110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21f303-412a-4039-b60d-20aafcf42d57 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:25.551843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 84/175] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:25.557409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1338edf-79f6-45d0-8429-eafbbc322a9e None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.558271 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1338edf-79f6-45d0-8429-eafbbc322a9e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.558557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1338edf-79f6-45d0-8429-eafbbc322a9e None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.559890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 85/176] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:25.561018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076f764-69a5-48ca-9f87-a9ec8d99afc8 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.561305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076f764-69a5-48ca-9f87-a9ec8d99afc8 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.561580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076f764-69a5-48ca-9f87-a9ec8d99afc8 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.562887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 86/177] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:25.567823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbf449-84ba-45b3-a713-3e1afb4ea26c None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.568113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbf449-84ba-45b3-a713-3e1afb4ea26c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.568393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbf449-84ba-45b3-a713-3e1afb4ea26c None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.569755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 87/178] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:25.570602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbf449-84ba-45b3-a713-3e1afb4ea26c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:25.599396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:25.603016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21f303-412a-4039-b60d-20aafcf42d57 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:25.636915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433add9-e14c-4ce8-9a8e-d4d4dc176521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:25.637572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433add9-e14c-4ce8-9a8e-d4d4dc176521 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.637858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433add9-e14c-4ce8-9a8e-d4d4dc176521 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.638142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433add9-e14c-4ce8-9a8e-d4d4dc176521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.640793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:25.683715 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbf449-84ba-45b3-a713-3e1afb4ea26c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:25.708331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433add9-e14c-4ce8-9a8e-d4d4dc176521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:25.709808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433add9-e14c-4ce8-9a8e-d4d4dc176521 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:25.719376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77ee03-485e-4632-9eb9-34b0d5392b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:25.720535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77ee03-485e-4632-9eb9-34b0d5392b5c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.720969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77ee03-485e-4632-9eb9-34b0d5392b5c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.721406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77ee03-485e-4632-9eb9-34b0d5392b5c None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.722576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77ee03-485e-4632-9eb9-34b0d5392b5c None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:25.724278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77ee03-485e-4632-9eb9-34b0d5392b5c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:25.731281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 88/179] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:25 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:25.731281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098127c7-0bb7-4d59-8c2a-a18ccf16703d None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.731281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098127c7-0bb7-4d59-8c2a-a18ccf16703d None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.731281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098127c7-0bb7-4d59-8c2a-a18ccf16703d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.745183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-098127c7-0bb7-4d59-8c2a-a18ccf16703d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:25.746352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-098127c7-0bb7-4d59-8c2a-a18ccf16703d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:25.817233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 89/180] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:25.818563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa5627-a383-4ad0-84e2-1d80319ebf03 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.818888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa5627-a383-4ad0-84e2-1d80319ebf03 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.819171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa5627-a383-4ad0-84e2-1d80319ebf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.837648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-42fa5627-a383-4ad0-84e2-1d80319ebf03 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:25.839371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-42fa5627-a383-4ad0-84e2-1d80319ebf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:25.842994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 92/181] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 15:36:25.844344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a6263-0973-4c7e-a091-05fbe691396d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:25.844636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a6263-0973-4c7e-a091-05fbe691396d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:25.844917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a6263-0973-4c7e-a091-05fbe691396d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:25.861995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:25.863891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-dd4a6263-0973-4c7e-a091-05fbe691396d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:25.865051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-dd4a6263-0973-4c7e-a091-05fbe691396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:25.893345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:25.917333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 90/182] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:25.918499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa5627-a383-4ad0-84e2-1d80319ebf03 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:25.934941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 93/183] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:25.943675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a6263-0973-4c7e-a091-05fbe691396d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.036780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa5627-a383-4ad0-84e2-1d80319ebf03 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:26.060238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a6263-0973-4c7e-a091-05fbe691396d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:26.072168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c5bc-352f-4d75-b66e-33657e625139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:26.072823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c5bc-352f-4d75-b66e-33657e625139 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.073127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c5bc-352f-4d75-b66e-33657e625139 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.073957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c5bc-352f-4d75-b66e-33657e625139 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.074825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c5bc-352f-4d75-b66e-33657e625139 None admin] RBAC: Adding query filter params (role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=54d37e7eb5474cf387beec6e56303b4f, user.id=4042a3d4d3d8483b9f9a6f1c750f1360) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:26.075149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c5bc-352f-4d75-b66e-33657e625139 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:26.076579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c5bc-352f-4d75-b66e-33657e625139 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:26.090754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 91/184] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:25 2022] GET /identity/v3/role_assignments?user.id=4042a3d4d3d8483b9f9a6f1c750f1360&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:26.090754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c5bc-352f-4d75-b66e-33657e625139 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.111033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:26.123056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117b555-95e4-4781-84f4-26800b3adc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:26.124965 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117b555-95e4-4781-84f4-26800b3adc99 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.125408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117b555-95e4-4781-84f4-26800b3adc99 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.125829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117b555-95e4-4781-84f4-26800b3adc99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.128196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117b555-95e4-4781-84f4-26800b3adc99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:26.129877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117b555-95e4-4781-84f4-26800b3adc99 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:26.138994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 94/185] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:26.140205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117b555-95e4-4781-84f4-26800b3adc99 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.177766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:26.228076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c5bc-352f-4d75-b66e-33657e625139 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:26.265134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117b555-95e4-4781-84f4-26800b3adc99 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:26.270423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2250b-70cd-4c1f-bda4-fa6b9d5a68a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:26.271455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2250b-70cd-4c1f-bda4-fa6b9d5a68a5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.271810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2250b-70cd-4c1f-bda4-fa6b9d5a68a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.272153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2250b-70cd-4c1f-bda4-fa6b9d5a68a5 None admin] PATH_INFO: `/v3/projects/0e9d0a88b7cf41aa8d2ad10351114364` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.273858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2250b-70cd-4c1f-bda4-fa6b9d5a68a5 None admin] RBAC: Authorizing `identity:get_project(project_id=0e9d0a88b7cf41aa8d2ad10351114364)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:26.275522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2250b-70cd-4c1f-bda4-fa6b9d5a68a5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:26.278145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 92/186] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:26 2022] GET /identity/v3/projects/0e9d0a88b7cf41aa8d2ad10351114364 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:26.278950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2250b-70cd-4c1f-bda4-fa6b9d5a68a5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.309359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0085be-d81e-42f1-9c2b-827adc884ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:26.310326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0085be-d81e-42f1-9c2b-827adc884ded None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.310951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0085be-d81e-42f1-9c2b-827adc884ded None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.311055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0085be-d81e-42f1-9c2b-827adc884ded None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.329343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0085be-d81e-42f1-9c2b-827adc884ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-6f0085be-d81e-42f1-9c2b-827adc884ded None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:26.337590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:26.338477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:26.339937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 95/187] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:26.341203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0085be-d81e-42f1-9c2b-827adc884ded None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.390265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:26.409352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2250b-70cd-4c1f-bda4-fa6b9d5a68a5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:26.447730 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:26.454347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18992cc8-385d-495c-be78-a66392d6e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:26.455369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18992cc8-385d-495c-be78-a66392d6e905 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.455796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18992cc8-385d-495c-be78-a66392d6e905 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.456269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18992cc8-385d-495c-be78-a66392d6e905 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.459308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0085be-d81e-42f1-9c2b-827adc884ded None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:26.468487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18992cc8-385d-495c-be78-a66392d6e905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-18992cc8-385d-495c-be78-a66392d6e905 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:26.476949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:26.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:26.479102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:26.481352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 93/188] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:26.483636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18992cc8-385d-495c-be78-a66392d6e905 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.495308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff91521-5c27-42cd-933b-dc423be7ea9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:26.495984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff91521-5c27-42cd-933b-dc423be7ea9f None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.496269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff91521-5c27-42cd-933b-dc423be7ea9f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.496548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff91521-5c27-42cd-933b-dc423be7ea9f None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.497260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff91521-5c27-42cd-933b-dc423be7ea9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:26.498716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff91521-5c27-42cd-933b-dc423be7ea9f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:26.518826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 96/189] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:36:26 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:26.519877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff91521-5c27-42cd-933b-dc423be7ea9f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.598446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18992cc8-385d-495c-be78-a66392d6e905 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:26.636132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7e9e0-422c-4f56-be5f-a6750ad543e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:26.636809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7e9e0-422c-4f56-be5f-a6750ad543e8 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.637109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7e9e0-422c-4f56-be5f-a6750ad543e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.637398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7e9e0-422c-4f56-be5f-a6750ad543e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.638312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7e9e0-422c-4f56-be5f-a6750ad543e8 None admin] RBAC: Adding query filter params (role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=0e9d0a88b7cf41aa8d2ad10351114364, user.id=4042a3d4d3d8483b9f9a6f1c750f1360) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:26.638748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7e9e0-422c-4f56-be5f-a6750ad543e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:26.640147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7e9e0-422c-4f56-be5f-a6750ad543e8 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:26.640833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff91521-5c27-42cd-933b-dc423be7ea9f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:26.654936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 94/190] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:26 2022] GET /identity/v3/role_assignments?user.id=4042a3d4d3d8483b9f9a6f1c750f1360&scope.project.id=0e9d0a88b7cf41aa8d2ad10351114364&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:26.654936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab5dc0-e488-4b88-aed7-e1171c5e190a None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.654936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab5dc0-e488-4b88-aed7-e1171c5e190a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.654936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab5dc0-e488-4b88-aed7-e1171c5e190a None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.654936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 95/191] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:26.654936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab5dc0-e488-4b88-aed7-e1171c5e190a None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.675160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:26.696480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca777fa8-3110-43d5-8d08-5aa51320a92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:26.697168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777fa8-3110-43d5-8d08-5aa51320a92b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.697446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777fa8-3110-43d5-8d08-5aa51320a92b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.697718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777fa8-3110-43d5-8d08-5aa51320a92b None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.698590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca777fa8-3110-43d5-8d08-5aa51320a92b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:26.699372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca777fa8-3110-43d5-8d08-5aa51320a92b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:26.700277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca777fa8-3110-43d5-8d08-5aa51320a92b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:26.709871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 97/192] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:26.711960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f4616-e9b7-442c-a401-1ddf76a90b83 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.712245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f4616-e9b7-442c-a401-1ddf76a90b83 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.712521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f4616-e9b7-442c-a401-1ddf76a90b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.733286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-895f4616-e9b7-442c-a401-1ddf76a90b83 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:26.734628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-895f4616-e9b7-442c-a401-1ddf76a90b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:26.738097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:26.802003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab5dc0-e488-4b88-aed7-e1171c5e190a None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:26.819390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 98/193] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:26.824807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f4616-e9b7-442c-a401-1ddf76a90b83 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.846807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023b409-8570-4e9c-a454-932189b1b982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:26.848159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023b409-8570-4e9c-a454-932189b1b982 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:26.848159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023b409-8570-4e9c-a454-932189b1b982 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:26.848323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023b409-8570-4e9c-a454-932189b1b982 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:26.848994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023b409-8570-4e9c-a454-932189b1b982 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:26.849729 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023b409-8570-4e9c-a454-932189b1b982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:26.851149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023b409-8570-4e9c-a454-932189b1b982 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:26.866504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 96/194] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:26.867745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023b409-8570-4e9c-a454-932189b1b982 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:26.968987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:26.969854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f4616-e9b7-442c-a401-1ddf76a90b83 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.010126 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023b409-8570-4e9c-a454-932189b1b982 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.016977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264c97f-0df1-4ac1-a088-9897025a071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.018657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264c97f-0df1-4ac1-a088-9897025a071e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.019191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264c97f-0df1-4ac1-a088-9897025a071e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.019666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264c97f-0df1-4ac1-a088-9897025a071e None admin] PATH_INFO: `/v3/users/8b12916f8b164b96830b6dc7892c2a47` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.034700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:27.040037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264c97f-0df1-4ac1-a088-9897025a071e None admin] RBAC: Authorizing `identity:get_user(user_id=8b12916f8b164b96830b6dc7892c2a47)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:27.042874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264c97f-0df1-4ac1-a088-9897025a071e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:27.045949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 99/195] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:26 2022] GET /identity/v3/users/8b12916f8b164b96830b6dc7892c2a47 => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:27.047024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264c97f-0df1-4ac1-a088-9897025a071e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:27.048814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750c876-8c1f-4aec-9b1a-b4b6549259e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.049947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750c876-8c1f-4aec-9b1a-b4b6549259e8 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.050423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750c876-8c1f-4aec-9b1a-b4b6549259e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.050983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750c876-8c1f-4aec-9b1a-b4b6549259e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.063761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750c876-8c1f-4aec-9b1a-b4b6549259e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-d750c876-8c1f-4aec-9b1a-b4b6549259e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:27.072328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:27.073758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:27.074545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:27.077160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 97/196] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:27.082607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750c876-8c1f-4aec-9b1a-b4b6549259e8 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:27.176106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264c97f-0df1-4ac1-a088-9897025a071e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.198407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750c876-8c1f-4aec-9b1a-b4b6549259e8 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.225858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af46445-1de2-4130-b0b9-e129770dec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.226931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af46445-1de2-4130-b0b9-e129770dec5c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.227223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af46445-1de2-4130-b0b9-e129770dec5c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.227503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af46445-1de2-4130-b0b9-e129770dec5c None admin] PATH_INFO: `/v3/users/8b12916f8b164b96830b6dc7892c2a47` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.229126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af46445-1de2-4130-b0b9-e129770dec5c None admin] RBAC: Authorizing `identity:get_user(user_id=8b12916f8b164b96830b6dc7892c2a47)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:27.231693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af46445-1de2-4130-b0b9-e129770dec5c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:27.234264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 100/197] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:27 2022] GET /identity/v3/users/8b12916f8b164b96830b6dc7892c2a47 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:27.235284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af46445-1de2-4130-b0b9-e129770dec5c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:27.236320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:27.243179 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2fbfb-a6a0-416a-8b90-0975b00ef52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.244111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2fbfb-a6a0-416a-8b90-0975b00ef52b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.244588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2fbfb-a6a0-416a-8b90-0975b00ef52b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.245609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2fbfb-a6a0-416a-8b90-0975b00ef52b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.318550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2fbfb-a6a0-416a-8b90-0975b00ef52b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:27.320425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2fbfb-a6a0-416a-8b90-0975b00ef52b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:27.320791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:27.365944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af46445-1de2-4130-b0b9-e129770dec5c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.411131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8334ee-8524-4012-a998-120402769f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.411807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8334ee-8524-4012-a998-120402769f1c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.412090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8334ee-8524-4012-a998-120402769f1c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.412369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8334ee-8524-4012-a998-120402769f1c None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.413281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8334ee-8524-4012-a998-120402769f1c None admin] RBAC: Adding query filter params (name=reader) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:27.413592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8334ee-8524-4012-a998-120402769f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:27.415293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8334ee-8524-4012-a998-120402769f1c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:27.426217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 101/198] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:27.430740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8334ee-8524-4012-a998-120402769f1c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:27.435277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 98/199] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:27.436333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2fbfb-a6a0-416a-8b90-0975b00ef52b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:27.471782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:27.555020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8334ee-8524-4012-a998-120402769f1c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.561938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2fbfb-a6a0-416a-8b90-0975b00ef52b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.583967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:27.594783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7277b65-5915-40d8-89ac-edc8422ae2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.595793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7277b65-5915-40d8-89ac-edc8422ae2b4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.596289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7277b65-5915-40d8-89ac-edc8422ae2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.596784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7277b65-5915-40d8-89ac-edc8422ae2b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.609744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d597c8-17c1-4547-880e-78651c38947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.611056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d597c8-17c1-4547-880e-78651c38947c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.611470 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d597c8-17c1-4547-880e-78651c38947c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.611891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d597c8-17c1-4547-880e-78651c38947c None admin] PATH_INFO: `/v3/users/338aa6519e7a497bb04fd0530823cf49` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.613114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d597c8-17c1-4547-880e-78651c38947c None admin] RBAC: Authorizing `identity:get_user(user_id=338aa6519e7a497bb04fd0530823cf49)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:27.615333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d597c8-17c1-4547-880e-78651c38947c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:27.617550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 99/200] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:27 2022] GET /identity/v3/users/338aa6519e7a497bb04fd0530823cf49 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:27.618857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d597c8-17c1-4547-880e-78651c38947c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:27.674326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7277b65-5915-40d8-89ac-edc8422ae2b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:27.676694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7277b65-5915-40d8-89ac-edc8422ae2b4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:27.745691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:27.749162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d597c8-17c1-4547-880e-78651c38947c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.790252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444500d-3f20-40d0-9d00-6843eb39b4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.791539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444500d-3f20-40d0-9d00-6843eb39b4ba None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.791855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444500d-3f20-40d0-9d00-6843eb39b4ba None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.792139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444500d-3f20-40d0-9d00-6843eb39b4ba None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.793585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444500d-3f20-40d0-9d00-6843eb39b4ba None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:27.795088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444500d-3f20-40d0-9d00-6843eb39b4ba None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:27.795728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 102/201] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:27.796759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7277b65-5915-40d8-89ac-edc8422ae2b4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:27.797161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 100/202] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:27 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:27.803702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444500d-3f20-40d0-9d00-6843eb39b4ba None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:27.834087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:27.920909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7277b65-5915-40d8-89ac-edc8422ae2b4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.935092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444500d-3f20-40d0-9d00-6843eb39b4ba None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:27.967691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a1c7b-dbf0-4987-b350-aeb370f4d23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.968625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a1c7b-dbf0-4987-b350-aeb370f4d23c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.968914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a1c7b-dbf0-4987-b350-aeb370f4d23c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.969218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a1c7b-dbf0-4987-b350-aeb370f4d23c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.973049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c8d98-66af-490a-8500-55773a969b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:27.974153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c8d98-66af-490a-8500-55773a969b5b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:27.974632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c8d98-66af-490a-8500-55773a969b5b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:27.975194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c8d98-66af-490a-8500-55773a969b5b None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:27.976645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c8d98-66af-490a-8500-55773a969b5b None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:27.978960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c8d98-66af-490a-8500-55773a969b5b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:27.981817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 101/203] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:27 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:27.983044 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c8d98-66af-490a-8500-55773a969b5b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:28.021354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:28.059101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a1c7b-dbf0-4987-b350-aeb370f4d23c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:28.060905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a1c7b-dbf0-4987-b350-aeb370f4d23c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:28.098115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:28.102433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c8d98-66af-490a-8500-55773a969b5b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:28.145846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334e54-271e-45a4-a94e-751b8ac1871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:28.146501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334e54-271e-45a4-a94e-751b8ac1871e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:28.146825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334e54-271e-45a4-a94e-751b8ac1871e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:28.147162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334e54-271e-45a4-a94e-751b8ac1871e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:28.148705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334e54-271e-45a4-a94e-751b8ac1871e None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c, user.id=8b12916f8b164b96830b6dc7892c2a47) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:28.149019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334e54-271e-45a4-a94e-751b8ac1871e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:28.150841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334e54-271e-45a4-a94e-751b8ac1871e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:28.163968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 102/204] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:27 2022] GET /identity/v3/role_assignments?user.id=8b12916f8b164b96830b6dc7892c2a47&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=433478cc318a467c931f13a757f161fa => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:28.163968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334e54-271e-45a4-a94e-751b8ac1871e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:28.201705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 103/205] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:28.207136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a1c7b-dbf0-4987-b350-aeb370f4d23c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:28.275905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:28.280083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334e54-271e-45a4-a94e-751b8ac1871e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:28.321418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2961d-11de-4468-94b3-93d1ec68dd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:28.322074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2961d-11de-4468-94b3-93d1ec68dd5b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:28.322364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2961d-11de-4468-94b3-93d1ec68dd5b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:28.322844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2961d-11de-4468-94b3-93d1ec68dd5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:28.326758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2961d-11de-4468-94b3-93d1ec68dd5b None admin] RBAC: Adding query filter params (role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c, user.id=8b12916f8b164b96830b6dc7892c2a47) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:28.326758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2961d-11de-4468-94b3-93d1ec68dd5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:28.326758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2961d-11de-4468-94b3-93d1ec68dd5b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:28.332744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 103/206] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:28 2022] GET /identity/v3/role_assignments?user.id=8b12916f8b164b96830b6dc7892c2a47&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:28.388327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a1c7b-dbf0-4987-b350-aeb370f4d23c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:28.424617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:28.440540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c743242-6248-44ef-829f-e020913a8d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:28.441437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c743242-6248-44ef-829f-e020913a8d55 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:28.441721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c743242-6248-44ef-829f-e020913a8d55 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:28.441999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c743242-6248-44ef-829f-e020913a8d55 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:28.443313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c743242-6248-44ef-829f-e020913a8d55 None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:28.444809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c743242-6248-44ef-829f-e020913a8d55 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:28.446959 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 104/207] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:28 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:28.456916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2961d-11de-4468-94b3-93d1ec68dd5b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:28.606135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2961d-11de-4468-94b3-93d1ec68dd5b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:28.651671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe6c37-f026-4741-ae16-b88f20eb3b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:28.652256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe6c37-f026-4741-ae16-b88f20eb3b80 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:28.652540 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe6c37-f026-4741-ae16-b88f20eb3b80 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:28.652813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe6c37-f026-4741-ae16-b88f20eb3b80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:28.653656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe6c37-f026-4741-ae16-b88f20eb3b80 None admin] RBAC: Adding query filter params (role.id=aab9a25c2ea047f89513ed57a120fa57, scope.project.id=54d37e7eb5474cf387beec6e56303b4f, user.id=338aa6519e7a497bb04fd0530823cf49) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:28.653969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe6c37-f026-4741-ae16-b88f20eb3b80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:28.655456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe6c37-f026-4741-ae16-b88f20eb3b80 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:28.667254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 104/208] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:28 2022] GET /identity/v3/role_assignments?user.id=338aa6519e7a497bb04fd0530823cf49&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=aab9a25c2ea047f89513ed57a120fa57 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:29.035284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dc75a-a9e3-4bb1-b459-a1717d3c1273 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.035773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dc75a-a9e3-4bb1-b459-a1717d3c1273 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.035773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2dc75a-a9e3-4bb1-b459-a1717d3c1273 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:29.037413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 105/209] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:29.041421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a6d40-b587-498a-b271-cf8b68d8e864 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.041766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a6d40-b587-498a-b271-cf8b68d8e864 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.042051 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a6d40-b587-498a-b271-cf8b68d8e864 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:29.043674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 105/210] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:29.044112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a215d-6080-4e23-9c1e-66dcc68590ae None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.044411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a215d-6080-4e23-9c1e-66dcc68590ae None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.044692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a215d-6080-4e23-9c1e-66dcc68590ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:29.051213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306ef0-3fbc-4cde-8ed6-c67b79d92820 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.051774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306ef0-3fbc-4cde-8ed6-c67b79d92820 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.052231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306ef0-3fbc-4cde-8ed6-c67b79d92820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:29.064091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-af2a215d-6080-4e23-9c1e-66dcc68590ae None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:29.065379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-af2a215d-6080-4e23-9c1e-66dcc68590ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:29.073875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d3306ef0-3fbc-4cde-8ed6-c67b79d92820 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:29.075292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d3306ef0-3fbc-4cde-8ed6-c67b79d92820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:29.102381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:29.158989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 106/211] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:29.161480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 106/212] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:29.185663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306ef0-3fbc-4cde-8ed6-c67b79d92820 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:29.191216 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a215d-6080-4e23-9c1e-66dcc68590ae None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:29.316120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a215d-6080-4e23-9c1e-66dcc68590ae None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:29.316560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306ef0-3fbc-4cde-8ed6-c67b79d92820 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:29.347297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:29.357370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f613a7-333c-49c9-bb81-8837b11f0bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:29.358168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f613a7-333c-49c9-bb81-8837b11f0bd4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.358458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f613a7-333c-49c9-bb81-8837b11f0bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.358763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f613a7-333c-49c9-bb81-8837b11f0bd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:29.360260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f613a7-333c-49c9-bb81-8837b11f0bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:29.361993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f613a7-333c-49c9-bb81-8837b11f0bd4 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:29.363967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 107/213] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:29.364479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ecc04-768d-4899-acdb-e1be6e0194b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:29.365551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecc04-768d-4899-acdb-e1be6e0194b2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.366053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecc04-768d-4899-acdb-e1be6e0194b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.366539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecc04-768d-4899-acdb-e1be6e0194b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:29.369617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f613a7-333c-49c9-bb81-8837b11f0bd4 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:29.380724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ecc04-768d-4899-acdb-e1be6e0194b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-224ecc04-768d-4899-acdb-e1be6e0194b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:29.391828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:29.392736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:29.394987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 107/214] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:29.399306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ecc04-768d-4899-acdb-e1be6e0194b2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:29.404194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:29.499536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f613a7-333c-49c9-bb81-8837b11f0bd4 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:29.516705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ecc04-768d-4899-acdb-e1be6e0194b2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:29.547488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a18860-369e-4309-af43-b73cff7aac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:29.549176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a18860-369e-4309-af43-b73cff7aac0b None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.549939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a18860-369e-4309-af43-b73cff7aac0b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.550524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a18860-369e-4309-af43-b73cff7aac0b None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:29.551753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a18860-369e-4309-af43-b73cff7aac0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:29.553572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96787b85-512f-44af-b1db-d9c54b7fcf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:29.553986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a18860-369e-4309-af43-b73cff7aac0b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:29.554241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96787b85-512f-44af-b1db-d9c54b7fcf7e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.554524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96787b85-512f-44af-b1db-d9c54b7fcf7e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.554839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96787b85-512f-44af-b1db-d9c54b7fcf7e None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:29.555592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96787b85-512f-44af-b1db-d9c54b7fcf7e None admin] RBAC: Adding query filter params (name=member) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:29.555899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96787b85-512f-44af-b1db-d9c54b7fcf7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:29.557781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96787b85-512f-44af-b1db-d9c54b7fcf7e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:29.568071 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 108/215] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:29.575372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96787b85-512f-44af-b1db-d9c54b7fcf7e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:29.582765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 108/216] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:36:29 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:36:29.653397 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:29.714536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96787b85-512f-44af-b1db-d9c54b7fcf7e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:29.753766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ed0f2-2ef0-4dee-8792-1d9ce705be8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:29.755020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ed0f2-2ef0-4dee-8792-1d9ce705be8b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.755513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ed0f2-2ef0-4dee-8792-1d9ce705be8b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.755991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ed0f2-2ef0-4dee-8792-1d9ce705be8b None admin] PATH_INFO: `/v3/users/c3f06405077e443887484a7f26a43f82` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:29.772090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ed0f2-2ef0-4dee-8792-1d9ce705be8b None admin] RBAC: Authorizing `identity:get_user(user_id=c3f06405077e443887484a7f26a43f82)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:29.774161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ed0f2-2ef0-4dee-8792-1d9ce705be8b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:29.776732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 109/217] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:29 2022] GET /identity/v3/users/c3f06405077e443887484a7f26a43f82 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:29.782969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a18860-369e-4309-af43-b73cff7aac0b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:29.874384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:29.894594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a18860-369e-4309-af43-b73cff7aac0b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:29.930491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a8af2-04c1-4a43-8c4f-cead7f0e54a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:29.931187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a8af2-04c1-4a43-8c4f-cead7f0e54a8 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:29.931474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a8af2-04c1-4a43-8c4f-cead7f0e54a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:29.931756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a8af2-04c1-4a43-8c4f-cead7f0e54a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:30.004523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a8af2-04c1-4a43-8c4f-cead7f0e54a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:30.006003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a8af2-04c1-4a43-8c4f-cead7f0e54a8 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:30.093428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:30.118040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 109/218] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 15:36:30.124763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ed0f2-2ef0-4dee-8792-1d9ce705be8b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:30.255410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ed0f2-2ef0-4dee-8792-1d9ce705be8b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:30.269179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:30.304579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7329a2-d9c4-48f7-97a5-8da52aed1d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:30.305813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7329a2-d9c4-48f7-97a5-8da52aed1d0d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:30.306295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7329a2-d9c4-48f7-97a5-8da52aed1d0d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:30.306814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7329a2-d9c4-48f7-97a5-8da52aed1d0d None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:30.311444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7329a2-d9c4-48f7-97a5-8da52aed1d0d None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:30.313111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7329a2-d9c4-48f7-97a5-8da52aed1d0d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:30.315074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 110/219] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:30 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:30.321365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a8af2-04c1-4a43-8c4f-cead7f0e54a8 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:30.469680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a8af2-04c1-4a43-8c4f-cead7f0e54a8 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:30.515156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768bd87-b204-46ae-9517-16444cc41b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:30.516052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768bd87-b204-46ae-9517-16444cc41b6b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:30.516511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768bd87-b204-46ae-9517-16444cc41b6b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:30.516947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768bd87-b204-46ae-9517-16444cc41b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:30.517958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768bd87-b204-46ae-9517-16444cc41b6b None admin] RBAC: Adding query filter params (role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c, user.id=c3f06405077e443887484a7f26a43f82) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:30.518415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768bd87-b204-46ae-9517-16444cc41b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:30.520020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768bd87-b204-46ae-9517-16444cc41b6b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:30.530984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 110/220] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:30 2022] GET /identity/v3/role_assignments?user.id=c3f06405077e443887484a7f26a43f82&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:30.762867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e737b-3b5a-4910-a9c9-4fcf82d25625 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:30.763241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e737b-3b5a-4910-a9c9-4fcf82d25625 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:30.763343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e737b-3b5a-4910-a9c9-4fcf82d25625 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:30.764985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 111/221] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:30.771743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44058bcc-8130-4e0c-a040-7513d7ffe6a5 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:30.772035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44058bcc-8130-4e0c-a040-7513d7ffe6a5 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:30.772313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44058bcc-8130-4e0c-a040-7513d7ffe6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:30.792968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-44058bcc-8130-4e0c-a040-7513d7ffe6a5 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:30.794486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-44058bcc-8130-4e0c-a040-7513d7ffe6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:30.841794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:30.880915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 111/222] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:30.906714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff8488-d596-4280-af80-210ec81e5093 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:30.907021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff8488-d596-4280-af80-210ec81e5093 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:30.907306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff8488-d596-4280-af80-210ec81e5093 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:30.908924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 112/223] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:30.914208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d62d1f-a1bf-48a0-ae71-f993d3d7b9f9 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:30.914753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d62d1f-a1bf-48a0-ae71-f993d3d7b9f9 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:30.915200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff8488-d596-4280-af80-210ec81e5093 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:30.915524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d62d1f-a1bf-48a0-ae71-f993d3d7b9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:30.933789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b1d62d1f-a1bf-48a0-ae71-f993d3d7b9f9 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:30.935356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b1d62d1f-a1bf-48a0-ae71-f993d3d7b9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:31.005003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 112/224] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:31.026856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d62d1f-a1bf-48a0-ae71-f993d3d7b9f9 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:31.098759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff8488-d596-4280-af80-210ec81e5093 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:31.131964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d62d1f-a1bf-48a0-ae71-f993d3d7b9f9 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:31.142023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:31.169597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4423a-0fb1-47ae-855f-c7a51b046616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:31.170825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4423a-0fb1-47ae-855f-c7a51b046616 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:31.171314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4423a-0fb1-47ae-855f-c7a51b046616 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:31.171742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4423a-0fb1-47ae-855f-c7a51b046616 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:31.179899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b39da-870f-4c49-8dee-5f385f6f31d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:31.183878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b39da-870f-4c49-8dee-5f385f6f31d6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:31.184234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b39da-870f-4c49-8dee-5f385f6f31d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:31.184998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4423a-0fb1-47ae-855f-c7a51b046616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:31.187076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b39da-870f-4c49-8dee-5f385f6f31d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-b2b4423a-0fb1-47ae-855f-c7a51b046616 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:31.194535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:31.195661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:31.196480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:31.199587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 113/225] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:31.206860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b39da-870f-4c49-8dee-5f385f6f31d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:31.217104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4423a-0fb1-47ae-855f-c7a51b046616 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-ec2b39da-870f-4c49-8dee-5f385f6f31d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:31.218134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:31.219057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:31.219858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 113/226] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:31.225501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b39da-870f-4c49-8dee-5f385f6f31d6 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:31.264523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:31.337767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b39da-870f-4c49-8dee-5f385f6f31d6 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:31.370709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4423a-0fb1-47ae-855f-c7a51b046616 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:31.375242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a2e4-5f58-44fa-b46d-0b0c99c971cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:31.375921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a2e4-5f58-44fa-b46d-0b0c99c971cf None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:31.376205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a2e4-5f58-44fa-b46d-0b0c99c971cf None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:31.376485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a2e4-5f58-44fa-b46d-0b0c99c971cf None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:31.377233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a2e4-5f58-44fa-b46d-0b0c99c971cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:31.377545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a2e4-5f58-44fa-b46d-0b0c99c971cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:31.379073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a2e4-5f58-44fa-b46d-0b0c99c971cf None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:31.388522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 114/227] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:31.394816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a2e4-5f58-44fa-b46d-0b0c99c971cf None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:31.414151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550b9d2-1ea0-496b-beea-6ad8758b397b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:31.414859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550b9d2-1ea0-496b-beea-6ad8758b397b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:31.415145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550b9d2-1ea0-496b-beea-6ad8758b397b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:31.415424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550b9d2-1ea0-496b-beea-6ad8758b397b None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:31.416189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550b9d2-1ea0-496b-beea-6ad8758b397b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:31.416525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550b9d2-1ea0-496b-beea-6ad8758b397b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:31.418482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550b9d2-1ea0-496b-beea-6ad8758b397b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:31.436266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 114/228] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:31.443702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550b9d2-1ea0-496b-beea-6ad8758b397b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:31.503732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:31.517318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:31.524093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a2e4-5f58-44fa-b46d-0b0c99c971cf None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:31.561272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d07fd-8efd-421b-a629-d2ba03925269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:31.562350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d07fd-8efd-421b-a629-d2ba03925269 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:31.562640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d07fd-8efd-421b-a629-d2ba03925269 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:31.562950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d07fd-8efd-421b-a629-d2ba03925269 None admin] PATH_INFO: `/v3/users/8b12916f8b164b96830b6dc7892c2a47` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:31.564163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d07fd-8efd-421b-a629-d2ba03925269 None admin] RBAC: Authorizing `identity:get_user(user_id=8b12916f8b164b96830b6dc7892c2a47)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:31.565732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d07fd-8efd-421b-a629-d2ba03925269 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:31.567429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 115/229] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:31 2022] GET /identity/v3/users/8b12916f8b164b96830b6dc7892c2a47 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:31.578717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d07fd-8efd-421b-a629-d2ba03925269 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:31.655389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550b9d2-1ea0-496b-beea-6ad8758b397b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:31.687571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d07fd-8efd-421b-a629-d2ba03925269 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:31.725106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcdf78-3dcf-4244-83fa-e7dcdd0bda5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:31.726140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcdf78-3dcf-4244-83fa-e7dcdd0bda5c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:31.726403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcdf78-3dcf-4244-83fa-e7dcdd0bda5c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:31.726748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcdf78-3dcf-4244-83fa-e7dcdd0bda5c None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:31.727449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff4ec8-9e24-4767-ac0c-84d3c5bd5a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:31.728544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff4ec8-9e24-4767-ac0c-84d3c5bd5a4b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:31.729541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff4ec8-9e24-4767-ac0c-84d3c5bd5a4b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:31.730035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff4ec8-9e24-4767-ac0c-84d3c5bd5a4b None admin] PATH_INFO: `/v3/users/8b12916f8b164b96830b6dc7892c2a47` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:31.736341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff4ec8-9e24-4767-ac0c-84d3c5bd5a4b None admin] RBAC: Authorizing `identity:get_user(user_id=8b12916f8b164b96830b6dc7892c2a47)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:31.736493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff4ec8-9e24-4767-ac0c-84d3c5bd5a4b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:31.738520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcdf78-3dcf-4244-83fa-e7dcdd0bda5c None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:31.740734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcdf78-3dcf-4244-83fa-e7dcdd0bda5c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:31.743827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 115/230] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:31 2022] GET /identity/v3/users/8b12916f8b164b96830b6dc7892c2a47 => generated 283 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:31.750201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 116/231] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:31 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:31.751207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff4ec8-9e24-4767-ac0c-84d3c5bd5a4b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:31.765061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcdf78-3dcf-4244-83fa-e7dcdd0bda5c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:31.818854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:31.951954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:31.960895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff4ec8-9e24-4767-ac0c-84d3c5bd5a4b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:31.979917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcdf78-3dcf-4244-83fa-e7dcdd0bda5c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:32.038010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333fbac-99b1-4432-bb7a-ab18483fd8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:32.038930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333fbac-99b1-4432-bb7a-ab18483fd8d8 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:32.039214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333fbac-99b1-4432-bb7a-ab18483fd8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:32.039490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333fbac-99b1-4432-bb7a-ab18483fd8d8 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/users/8b12916f8b164b96830b6dc7892c2a47/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:32.042856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333fbac-99b1-4432-bb7a-ab18483fd8d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fc1137956e4fdb9e9aeb9740c88d9c, user_id=8b12916f8b164b96830b6dc7892c2a47, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:32.044325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333fbac-99b1-4432-bb7a-ab18483fd8d8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:32.047884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b844c-d4c4-440c-8039-1d085eee0356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:32.048756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b844c-d4c4-440c-8039-1d085eee0356 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:32.049034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b844c-d4c4-440c-8039-1d085eee0356 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:32.049314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b844c-d4c4-440c-8039-1d085eee0356 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:32.050523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b844c-d4c4-440c-8039-1d085eee0356 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:32.051985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b844c-d4c4-440c-8039-1d085eee0356 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:32.053999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 116/232] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:31 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:32.060857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b844c-d4c4-440c-8039-1d085eee0356 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:32.081777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 117/233] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:36:31 2022] PUT /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/users/8b12916f8b164b96830b6dc7892c2a47/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:32.204360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b844c-d4c4-440c-8039-1d085eee0356 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:32.250456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df385e4d-6ea6-450a-9c06-e55c3f5cbae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:32.251394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df385e4d-6ea6-450a-9c06-e55c3f5cbae1 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:32.251681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df385e4d-6ea6-450a-9c06-e55c3f5cbae1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:32.251960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df385e4d-6ea6-450a-9c06-e55c3f5cbae1 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/users/8b12916f8b164b96830b6dc7892c2a47/roles/7fb980f85eb04293a0d1dba515bf0a2c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:32.254234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df385e4d-6ea6-450a-9c06-e55c3f5cbae1 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fc1137956e4fdb9e9aeb9740c88d9c, user_id=8b12916f8b164b96830b6dc7892c2a47, role_id=7fb980f85eb04293a0d1dba515bf0a2c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:32.255705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df385e4d-6ea6-450a-9c06-e55c3f5cbae1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:32.272453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f552d-903b-4e64-aaa8-3276ddc6352e None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:32.273383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f552d-903b-4e64-aaa8-3276ddc6352e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:32.273383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f552d-903b-4e64-aaa8-3276ddc6352e None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:32.276814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 117/235] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:36:32 2022] PUT /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/users/8b12916f8b164b96830b6dc7892c2a47/roles/7fb980f85eb04293a0d1dba515bf0a2c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:32.278674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 118/235] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:36:32.279997 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83cdba-388c-46cf-b8b7-7c99d67ea908 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:32.280289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83cdba-388c-46cf-b8b7-7c99d67ea908 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:32.280575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83cdba-388c-46cf-b8b7-7c99d67ea908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:32.312364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8a83cdba-388c-46cf-b8b7-7c99d67ea908 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:32.313620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8a83cdba-388c-46cf-b8b7-7c99d67ea908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:32.339126 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:32.413494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 118/236] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:32.438057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f552d-903b-4e64-aaa8-3276ddc6352e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:32.549599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:32.562899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f552d-903b-4e64-aaa8-3276ddc6352e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:32.598809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77354052-c581-42d5-9979-c706c1d302c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:32.599617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77354052-c581-42d5-9979-c706c1d302c9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:32.599902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77354052-c581-42d5-9979-c706c1d302c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:32.600183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77354052-c581-42d5-9979-c706c1d302c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:32.612423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77354052-c581-42d5-9979-c706c1d302c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-77354052-c581-42d5-9979-c706c1d302c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:32.620980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:32.621878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:32.622668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 119/237] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:32.628490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83cdba-388c-46cf-b8b7-7c99d67ea908 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:32.774224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94debbc3-964d-43f5-98d1-7d2917d26f1b None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:32.774816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94debbc3-964d-43f5-98d1-7d2917d26f1b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:32.775388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94debbc3-964d-43f5-98d1-7d2917d26f1b None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:32.777856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 120/238] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:32.786292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea0597-0e74-4145-bc8b-79fce5ab868e None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:32.786666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea0597-0e74-4145-bc8b-79fce5ab868e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:32.786994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea0597-0e74-4145-bc8b-79fce5ab868e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:32.788517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83cdba-388c-46cf-b8b7-7c99d67ea908 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:32.819882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-8bea0597-0e74-4145-bc8b-79fce5ab868e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:32.821010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8bea0597-0e74-4145-bc8b-79fce5ab868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:32.838843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15a72-2836-434c-b491-2edee3c482b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:32.839525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15a72-2836-434c-b491-2edee3c482b6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:32.839805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15a72-2836-434c-b491-2edee3c482b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:32.840079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15a72-2836-434c-b491-2edee3c482b6 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:32.842884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a15a72-2836-434c-b491-2edee3c482b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:32.843203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a15a72-2836-434c-b491-2edee3c482b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:32.844587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a15a72-2836-434c-b491-2edee3c482b6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:32.866658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 119/239] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:32.868504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15a72-2836-434c-b491-2edee3c482b6 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:32.909371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:32.961113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 121/240] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:33.005526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea0597-0e74-4145-bc8b-79fce5ab868e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:33.007808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15a72-2836-434c-b491-2edee3c482b6 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:33.063445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c5603-7092-4a98-b531-a5f6283ab031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:33.064339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c5603-7092-4a98-b531-a5f6283ab031 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:33.064626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c5603-7092-4a98-b531-a5f6283ab031 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:33.064907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c5603-7092-4a98-b531-a5f6283ab031 None admin] PATH_INFO: `/v3/users/631d860d924d45b69d717f079ce72c24` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:33.080038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c5603-7092-4a98-b531-a5f6283ab031 None admin] RBAC: Authorizing `identity:get_user(user_id=631d860d924d45b69d717f079ce72c24)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:33.081500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c5603-7092-4a98-b531-a5f6283ab031 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:33.083318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 120/241] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:32 2022] GET /identity/v3/users/631d860d924d45b69d717f079ce72c24 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:33.088846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c5603-7092-4a98-b531-a5f6283ab031 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:33.143193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:33.175132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:33.179411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea0597-0e74-4145-bc8b-79fce5ab868e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:33.202456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c5603-7092-4a98-b531-a5f6283ab031 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:33.236123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bce075-40ba-4246-9ed7-0a2e6d15df75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:33.237255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bce075-40ba-4246-9ed7-0a2e6d15df75 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:33.237780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bce075-40ba-4246-9ed7-0a2e6d15df75 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:33.238213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bce075-40ba-4246-9ed7-0a2e6d15df75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:33.239063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d48ca7-8460-4cc9-a27a-894e63f0bce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:33.239400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bce075-40ba-4246-9ed7-0a2e6d15df75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:33.240181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d48ca7-8460-4cc9-a27a-894e63f0bce4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:33.240666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d48ca7-8460-4cc9-a27a-894e63f0bce4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:33.240860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bce075-40ba-4246-9ed7-0a2e6d15df75 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:33.241348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d48ca7-8460-4cc9-a27a-894e63f0bce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:33.242976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 122/242] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:33.244220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1928d-6d94-438d-8757-dcf75f5e7df5 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:33.244542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1928d-6d94-438d-8757-dcf75f5e7df5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:33.244996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1928d-6d94-438d-8757-dcf75f5e7df5 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:33.246660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 123/243] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:33.250923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1928d-6d94-438d-8757-dcf75f5e7df5 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:33.315245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d48ca7-8460-4cc9-a27a-894e63f0bce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:33.317132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d48ca7-8460-4cc9-a27a-894e63f0bce4 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:33.361524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1928d-6d94-438d-8757-dcf75f5e7df5 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:33.395472 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:33.400187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53293eed-a5b1-4335-bed8-8641a402238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:33.401439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293eed-a5b1-4335-bed8-8641a402238a None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:33.402411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293eed-a5b1-4335-bed8-8641a402238a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:33.402691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293eed-a5b1-4335-bed8-8641a402238a None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:33.403425 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53293eed-a5b1-4335-bed8-8641a402238a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:33.405329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53293eed-a5b1-4335-bed8-8641a402238a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:33.432548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 121/244] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 15:36:33.433063 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 124/245] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:36:33 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:33.434314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d48ca7-8460-4cc9-a27a-894e63f0bce4 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:33.434689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fba3e-c39a-4243-b80b-eecbfc49bcad None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:33.435263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fba3e-c39a-4243-b80b-eecbfc49bcad None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:33.435836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fba3e-c39a-4243-b80b-eecbfc49bcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:33.459963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:33.469683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-140fba3e-c39a-4243-b80b-eecbfc49bcad None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:33.472067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-140fba3e-c39a-4243-b80b-eecbfc49bcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:33.559513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 125/246] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:33.592648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d48ca7-8460-4cc9-a27a-894e63f0bce4 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:33.638482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fba3e-c39a-4243-b80b-eecbfc49bcad None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:33.641572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d97cd-a2ef-485e-9d67-b7cc5134929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:33.642459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d97cd-a2ef-485e-9d67-b7cc5134929a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:33.642777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d97cd-a2ef-485e-9d67-b7cc5134929a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:33.643037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d97cd-a2ef-485e-9d67-b7cc5134929a None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:33.645221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d97cd-a2ef-485e-9d67-b7cc5134929a None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:33.646703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d97cd-a2ef-485e-9d67-b7cc5134929a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:33.649127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 122/247] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:33 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:33.655192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d97cd-a2ef-485e-9d67-b7cc5134929a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:33.738900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:33.804604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fba3e-c39a-4243-b80b-eecbfc49bcad None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:33.811247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d97cd-a2ef-485e-9d67-b7cc5134929a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:33.847024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888363d-19bd-49d7-b710-40847ca7c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:33.847922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888363d-19bd-49d7-b710-40847ca7c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:33.848209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888363d-19bd-49d7-b710-40847ca7c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:33.848482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888363d-19bd-49d7-b710-40847ca7c3b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:33.853372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:33.854385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691cdbb-cf90-464a-82d2-8af58e3bbdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:33.855192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691cdbb-cf90-464a-82d2-8af58e3bbdbf None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:33.855561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691cdbb-cf90-464a-82d2-8af58e3bbdbf None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:33.855840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691cdbb-cf90-464a-82d2-8af58e3bbdbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:33.856683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691cdbb-cf90-464a-82d2-8af58e3bbdbf None admin] RBAC: Adding query filter params (role.id=aab9a25c2ea047f89513ed57a120fa57, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c, user.id=631d860d924d45b69d717f079ce72c24) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:33.857139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691cdbb-cf90-464a-82d2-8af58e3bbdbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:33.859604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691cdbb-cf90-464a-82d2-8af58e3bbdbf None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:33.865795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888363d-19bd-49d7-b710-40847ca7c3b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:33.868296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 123/248] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:33 2022] GET /identity/v3/role_assignments?user.id=631d860d924d45b69d717f079ce72c24&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=aab9a25c2ea047f89513ed57a120fa57 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-7888363d-19bd-49d7-b710-40847ca7c3b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:33.875646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:33.876711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:33.877707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:33.884872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 126/249] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:33.887196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691cdbb-cf90-464a-82d2-8af58e3bbdbf None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:34.007522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691cdbb-cf90-464a-82d2-8af58e3bbdbf None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:34.016559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:34.049788 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b1bab-1587-486e-82f1-da094de71446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:34.050480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b1bab-1587-486e-82f1-da094de71446 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:34.050778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b1bab-1587-486e-82f1-da094de71446 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:34.051057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b1bab-1587-486e-82f1-da094de71446 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:34.051817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b1bab-1587-486e-82f1-da094de71446 None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:34.052123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b1bab-1587-486e-82f1-da094de71446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:34.053512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b1bab-1587-486e-82f1-da094de71446 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:34.065917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 124/250] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:34.071590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888363d-19bd-49d7-b710-40847ca7c3b8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:34.186320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888363d-19bd-49d7-b710-40847ca7c3b8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:34.226578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661f309-a6dc-4338-b4f7-0d3d3ed70554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:34.227873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661f309-a6dc-4338-b4f7-0d3d3ed70554 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:34.228393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661f309-a6dc-4338-b4f7-0d3d3ed70554 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:34.228929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661f309-a6dc-4338-b4f7-0d3d3ed70554 None admin] PATH_INFO: `/v3/users/c3f06405077e443887484a7f26a43f82` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:34.230892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661f309-a6dc-4338-b4f7-0d3d3ed70554 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f06405077e443887484a7f26a43f82)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:34.232600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661f309-a6dc-4338-b4f7-0d3d3ed70554 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:34.235635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 127/251] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:34 2022] GET /identity/v3/users/c3f06405077e443887484a7f26a43f82 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:34.240062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b1bab-1587-486e-82f1-da094de71446 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:34.355149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b1bab-1587-486e-82f1-da094de71446 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:34.390591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a96f4d-e502-4fad-9b2e-f8d305100558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:34.391531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96f4d-e502-4fad-9b2e-f8d305100558 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:34.391934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96f4d-e502-4fad-9b2e-f8d305100558 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:34.392215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96f4d-e502-4fad-9b2e-f8d305100558 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:34.393427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a96f4d-e502-4fad-9b2e-f8d305100558 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:34.395003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a96f4d-e502-4fad-9b2e-f8d305100558 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:34.400061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 125/252] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:34 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:34.404439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661f309-a6dc-4338-b4f7-0d3d3ed70554 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:34.503258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:34.546316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661f309-a6dc-4338-b4f7-0d3d3ed70554 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:34.590301 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f7a38-c26a-4e77-9918-632b16a483cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:34.591234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7a38-c26a-4e77-9918-632b16a483cd None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:34.591522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7a38-c26a-4e77-9918-632b16a483cd None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:34.591803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7a38-c26a-4e77-9918-632b16a483cd None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/users/c3f06405077e443887484a7f26a43f82/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:34.600083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f7a38-c26a-4e77-9918-632b16a483cd None admin] RBAC: Authorizing `identity:create_grant(project_id=25fc1137956e4fdb9e9aeb9740c88d9c, user_id=c3f06405077e443887484a7f26a43f82, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:34.601556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f7a38-c26a-4e77-9918-632b16a483cd None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:34.618548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 128/253] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:36:34 2022] PUT /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/users/c3f06405077e443887484a7f26a43f82/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:34.706173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cff589-353c-46b0-b666-c1f04fed5c83 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:34.707287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cff589-353c-46b0-b666-c1f04fed5c83 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:34.707766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cff589-353c-46b0-b666-c1f04fed5c83 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:34.709647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 126/254] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:34.715611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb97296-bd99-43a8-9734-3adb941a7809 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:34.715952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb97296-bd99-43a8-9734-3adb941a7809 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:34.716249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb97296-bd99-43a8-9734-3adb941a7809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:34.737227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-3eb97296-bd99-43a8-9734-3adb941a7809 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:34.738460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3eb97296-bd99-43a8-9734-3adb941a7809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:34.826526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 129/255] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:34.870393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bd378-fb54-4970-94f0-951968997bbe None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:34.870684 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bd378-fb54-4970-94f0-951968997bbe None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:34.870932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bd378-fb54-4970-94f0-951968997bbe None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:34.872536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 127/256] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:34.876585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb97296-bd99-43a8-9734-3adb941a7809 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:34.880943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87608716-1235-4be3-8ea1-4a8e2b084a7e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:34.881232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87608716-1235-4be3-8ea1-4a8e2b084a7e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:34.881514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87608716-1235-4be3-8ea1-4a8e2b084a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:34.903063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-87608716-1235-4be3-8ea1-4a8e2b084a7e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:34.904286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-87608716-1235-4be3-8ea1-4a8e2b084a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:34.954098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:34.989985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:35.005866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 128/257] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:35.035065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87608716-1235-4be3-8ea1-4a8e2b084a7e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:35.044673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb97296-bd99-43a8-9734-3adb941a7809 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:35.085169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772446c-7a85-49d6-92c4-5908ec4ae165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:35.086047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772446c-7a85-49d6-92c4-5908ec4ae165 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:35.086328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772446c-7a85-49d6-92c4-5908ec4ae165 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:35.086602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772446c-7a85-49d6-92c4-5908ec4ae165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:35.099088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772446c-7a85-49d6-92c4-5908ec4ae165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-d772446c-7a85-49d6-92c4-5908ec4ae165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:35.108182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:35.109058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:35.109867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 130/258] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:35.119035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772446c-7a85-49d6-92c4-5908ec4ae165 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:35.238350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772446c-7a85-49d6-92c4-5908ec4ae165 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:35.254215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:35.278093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdab248-2a8b-4273-8fbd-13cc8f3cabda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:35.279206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdab248-2a8b-4273-8fbd-13cc8f3cabda None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:35.279632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdab248-2a8b-4273-8fbd-13cc8f3cabda None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:35.280050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdab248-2a8b-4273-8fbd-13cc8f3cabda None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:35.281210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdab248-2a8b-4273-8fbd-13cc8f3cabda None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:35.281658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdab248-2a8b-4273-8fbd-13cc8f3cabda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:35.283930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdab248-2a8b-4273-8fbd-13cc8f3cabda None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:35.286813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87608716-1235-4be3-8ea1-4a8e2b084a7e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:35.295255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 131/259] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:35.303977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdab248-2a8b-4273-8fbd-13cc8f3cabda None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:35.335558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49c3e7-05da-495b-afb1-88cf093b7eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:35.336387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49c3e7-05da-495b-afb1-88cf093b7eaf None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:35.336675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49c3e7-05da-495b-afb1-88cf093b7eaf None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:35.336956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49c3e7-05da-495b-afb1-88cf093b7eaf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:35.354624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49c3e7-05da-495b-afb1-88cf093b7eaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-6f49c3e7-05da-495b-afb1-88cf093b7eaf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:35.372106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:35.373239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:35.374221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 129/260] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:35.381062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49c3e7-05da-495b-afb1-88cf093b7eaf None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:35.420953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdab248-2a8b-4273-8fbd-13cc8f3cabda None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:35.425398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:35.464788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4a6c0-6e7d-48ff-90ad-7e269858faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:35.465881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4a6c0-6e7d-48ff-90ad-7e269858faba None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:35.466305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4a6c0-6e7d-48ff-90ad-7e269858faba None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:35.466719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4a6c0-6e7d-48ff-90ad-7e269858faba None admin] PATH_INFO: `/v3/users/8b12916f8b164b96830b6dc7892c2a47` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:35.468095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4a6c0-6e7d-48ff-90ad-7e269858faba None admin] RBAC: Authorizing `identity:get_user(user_id=8b12916f8b164b96830b6dc7892c2a47)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:35.469677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4a6c0-6e7d-48ff-90ad-7e269858faba None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:35.471805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 132/261] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:35 2022] GET /identity/v3/users/8b12916f8b164b96830b6dc7892c2a47 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:35.480187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4a6c0-6e7d-48ff-90ad-7e269858faba None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:35.540376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:35.559839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49c3e7-05da-495b-afb1-88cf093b7eaf None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:35.601008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4a6c0-6e7d-48ff-90ad-7e269858faba None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:35.618411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d1938-c3b3-4a8d-9cb4-5c5b42d96a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:35.619146 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d1938-c3b3-4a8d-9cb4-5c5b42d96a43 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:35.619608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d1938-c3b3-4a8d-9cb4-5c5b42d96a43 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:35.619844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d1938-c3b3-4a8d-9cb4-5c5b42d96a43 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:35.620771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d1938-c3b3-4a8d-9cb4-5c5b42d96a43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:35.621093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d1938-c3b3-4a8d-9cb4-5c5b42d96a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:35.622620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d1938-c3b3-4a8d-9cb4-5c5b42d96a43 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:35.638163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 130/262] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:35.642095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10ceba-fb90-4aea-96db-cd8fbb52e7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:35.643131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10ceba-fb90-4aea-96db-cd8fbb52e7d4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:35.643562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d1938-c3b3-4a8d-9cb4-5c5b42d96a43 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:35.643825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10ceba-fb90-4aea-96db-cd8fbb52e7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:35.644309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10ceba-fb90-4aea-96db-cd8fbb52e7d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:35.718234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10ceba-fb90-4aea-96db-cd8fbb52e7d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:35.720308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10ceba-fb90-4aea-96db-cd8fbb52e7d4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:35.794146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:35.801120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:35.824209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d1938-c3b3-4a8d-9cb4-5c5b42d96a43 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:35.907354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 133/263] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:35.909008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37d78a-b640-4ed0-8f5a-54af5c161496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:35.909973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37d78a-b640-4ed0-8f5a-54af5c161496 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:35.910257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37d78a-b640-4ed0-8f5a-54af5c161496 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:35.910537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37d78a-b640-4ed0-8f5a-54af5c161496 None admin] PATH_INFO: `/v3/users/8b12916f8b164b96830b6dc7892c2a47` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:35.911796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37d78a-b640-4ed0-8f5a-54af5c161496 None admin] RBAC: Authorizing `identity:get_user(user_id=8b12916f8b164b96830b6dc7892c2a47)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:35.913427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10ceba-fb90-4aea-96db-cd8fbb52e7d4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:35.923183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37d78a-b640-4ed0-8f5a-54af5c161496 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:35.924933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 131/264] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:35 2022] GET /identity/v3/users/8b12916f8b164b96830b6dc7892c2a47 => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:35.930786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37d78a-b640-4ed0-8f5a-54af5c161496 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:36.067096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10ceba-fb90-4aea-96db-cd8fbb52e7d4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:36.091675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37d78a-b640-4ed0-8f5a-54af5c161496 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:36.101977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931223d1-db7d-4307-95a3-04482de7bb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:36.106753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931223d1-db7d-4307-95a3-04482de7bb22 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:36.106753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931223d1-db7d-4307-95a3-04482de7bb22 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:36.106753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931223d1-db7d-4307-95a3-04482de7bb22 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:36.106753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931223d1-db7d-4307-95a3-04482de7bb22 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:36.106753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931223d1-db7d-4307-95a3-04482de7bb22 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:36.110545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 134/265] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:35 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:36.115173 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931223d1-db7d-4307-95a3-04482de7bb22 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:36.143083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:36.161996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95f71-b08a-4466-94cb-15ca0c24491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:36.162696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95f71-b08a-4466-94cb-15ca0c24491f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:36.163008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95f71-b08a-4466-94cb-15ca0c24491f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:36.163287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95f71-b08a-4466-94cb-15ca0c24491f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:36.206657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:36.279804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a95f71-b08a-4466-94cb-15ca0c24491f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:36.281620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a95f71-b08a-4466-94cb-15ca0c24491f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:36.374560 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931223d1-db7d-4307-95a3-04482de7bb22 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:36.394286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 132/266] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 15:36:36.400799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95f71-b08a-4466-94cb-15ca0c24491f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:36.485582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:36.492057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdc29c-2b06-4a93-94d7-1ab4ec66dcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:36.492836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdc29c-2b06-4a93-94d7-1ab4ec66dcb6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:36.493173 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdc29c-2b06-4a93-94d7-1ab4ec66dcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:36.493673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdc29c-2b06-4a93-94d7-1ab4ec66dcb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:36.494839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdc29c-2b06-4a93-94d7-1ab4ec66dcb6 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c, user.id=8b12916f8b164b96830b6dc7892c2a47) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:36.497261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdc29c-2b06-4a93-94d7-1ab4ec66dcb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:36.498788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdc29c-2b06-4a93-94d7-1ab4ec66dcb6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:36.514177 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 135/267] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:36 2022] GET /identity/v3/role_assignments?user.id=8b12916f8b164b96830b6dc7892c2a47&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=433478cc318a467c931f13a757f161fa => generated 608 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:36.544543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95f71-b08a-4466-94cb-15ca0c24491f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:36.596854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32f9ef-0327-47e9-9a52-dac4b4fb8858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:36.598112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32f9ef-0327-47e9-9a52-dac4b4fb8858 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:36.598587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32f9ef-0327-47e9-9a52-dac4b4fb8858 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:36.599114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32f9ef-0327-47e9-9a52-dac4b4fb8858 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:36.600560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32f9ef-0327-47e9-9a52-dac4b4fb8858 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:36.602295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32f9ef-0327-47e9-9a52-dac4b4fb8858 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:36.604412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 133/268] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:36 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:36.611176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdc29c-2b06-4a93-94d7-1ab4ec66dcb6 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:36.749540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:36.784468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdc29c-2b06-4a93-94d7-1ab4ec66dcb6 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:36.821740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ae8a6-9dca-4629-a2a5-d4e539b02b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:36.822474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ae8a6-9dca-4629-a2a5-d4e539b02b70 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:36.822835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ae8a6-9dca-4629-a2a5-d4e539b02b70 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:36.823124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ae8a6-9dca-4629-a2a5-d4e539b02b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:36.823956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ae8a6-9dca-4629-a2a5-d4e539b02b70 None admin] RBAC: Adding query filter params (role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c, user.id=8b12916f8b164b96830b6dc7892c2a47) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:36.824263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ae8a6-9dca-4629-a2a5-d4e539b02b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:36.825712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ae8a6-9dca-4629-a2a5-d4e539b02b70 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:36.832433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 136/269] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:36 2022] GET /identity/v3/role_assignments?user.id=8b12916f8b164b96830b6dc7892c2a47&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:37.096417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f749-2745-4afa-a33a-1dc1401f7e7e None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.096891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f749-2745-4afa-a33a-1dc1401f7e7e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.096891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f749-2745-4afa-a33a-1dc1401f7e7e None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.098466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 134/270] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:37.104974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b3f30-c623-4e5a-9808-9b24960c78da None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.105300 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b3f30-c623-4e5a-9808-9b24960c78da None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.105538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b3f30-c623-4e5a-9808-9b24960c78da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.127217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e26b3f30-c623-4e5a-9808-9b24960c78da None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:37.128423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e26b3f30-c623-4e5a-9808-9b24960c78da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:37.233483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 137/271] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:37.267363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488f749-2745-4afa-a33a-1dc1401f7e7e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:37.397050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488f749-2745-4afa-a33a-1dc1401f7e7e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:37.425962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:37.438861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbd360-5fe1-4422-ba52-740c116b1aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:37.439700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbd360-5fe1-4422-ba52-740c116b1aa4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.439980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbd360-5fe1-4422-ba52-740c116b1aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.440606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbd360-5fe1-4422-ba52-740c116b1aa4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.454432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbd360-5fe1-4422-ba52-740c116b1aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-0afbd360-5fe1-4422-ba52-740c116b1aa4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:37.463566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:37.464461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:37.465692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 135/272] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:37.475133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b3f30-c623-4e5a-9808-9b24960c78da None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:37.494804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eadcb4-abc1-4dd3-a0b4-70a04df7cdaa None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.495027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eadcb4-abc1-4dd3-a0b4-70a04df7cdaa None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.495318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eadcb4-abc1-4dd3-a0b4-70a04df7cdaa None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.497032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 136/273] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:36:37.502447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e17ac4-1c3d-4b00-b89e-cf80209acb42 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.502833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e17ac4-1c3d-4b00-b89e-cf80209acb42 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.503098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e17ac4-1c3d-4b00-b89e-cf80209acb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.521109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b5e17ac4-1c3d-4b00-b89e-cf80209acb42 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:37.522574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b5e17ac4-1c3d-4b00-b89e-cf80209acb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:37.564976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:37.602673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b3f30-c623-4e5a-9808-9b24960c78da None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:37.607965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 137/274] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:37.623193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3276a2-2719-4e69-81ab-ef6ec55b67d0 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.624227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3276a2-2719-4e69-81ab-ef6ec55b67d0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.624882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3276a2-2719-4e69-81ab-ef6ec55b67d0 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.627011 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 138/275] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:36:37.632039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8388bf-8ab7-49ae-9215-7940260dea71 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.633006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8388bf-8ab7-49ae-9215-7940260dea71 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.633298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8388bf-8ab7-49ae-9215-7940260dea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.647748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e83d4-0976-416b-ab0f-9cf9afbaaa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:37.648353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e83d4-0976-416b-ab0f-9cf9afbaaa5c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.648638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e83d4-0976-416b-ab0f-9cf9afbaaa5c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.648914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e83d4-0976-416b-ab0f-9cf9afbaaa5c None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.649654 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e83d4-0976-416b-ab0f-9cf9afbaaa5c None admin] RBAC: Adding query filter params (name=reader) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:37.649962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e83d4-0976-416b-ab0f-9cf9afbaaa5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:37.652014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e83d4-0976-416b-ab0f-9cf9afbaaa5c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:37.652668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-2d8388bf-8ab7-49ae-9215-7940260dea71 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:37.654387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2d8388bf-8ab7-49ae-9215-7940260dea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:37.661772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 138/276] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:37.663238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e83d4-0976-416b-ab0f-9cf9afbaaa5c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:37.737575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 139/277] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:37.738498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8388bf-8ab7-49ae-9215-7940260dea71 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:37.776165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e83d4-0976-416b-ab0f-9cf9afbaaa5c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:37.787224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:37.809365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:37.816878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01e595-d67c-4ff1-9aaf-9bafe9086074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:37.817777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01e595-d67c-4ff1-9aaf-9bafe9086074 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.818115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01e595-d67c-4ff1-9aaf-9bafe9086074 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.818412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01e595-d67c-4ff1-9aaf-9bafe9086074 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.819747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01e595-d67c-4ff1-9aaf-9bafe9086074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:37.821923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01e595-d67c-4ff1-9aaf-9bafe9086074 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:37.828936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 139/278] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:37.828936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01e595-d67c-4ff1-9aaf-9bafe9086074 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:37.849043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8388bf-8ab7-49ae-9215-7940260dea71 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:37.884897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25996c-8162-4bed-b221-2e942c295d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:37.886272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25996c-8162-4bed-b221-2e942c295d90 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.886553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25996c-8162-4bed-b221-2e942c295d90 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.886856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25996c-8162-4bed-b221-2e942c295d90 None admin] PATH_INFO: `/v3/users/631d860d924d45b69d717f079ce72c24` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.888046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25996c-8162-4bed-b221-2e942c295d90 None admin] RBAC: Authorizing `identity:get_user(user_id=631d860d924d45b69d717f079ce72c24)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:37.889507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25996c-8162-4bed-b221-2e942c295d90 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:37.895805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 140/279] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:37 2022] GET /identity/v3/users/631d860d924d45b69d717f079ce72c24 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:37.895805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25996c-8162-4bed-b221-2e942c295d90 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:37.932241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01e595-d67c-4ff1-9aaf-9bafe9086074 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:37.965140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48814d-1532-4750-a246-2e2881a6be36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:37.965933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48814d-1532-4750-a246-2e2881a6be36 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.966209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48814d-1532-4750-a246-2e2881a6be36 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.966482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48814d-1532-4750-a246-2e2881a6be36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.976302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48814d-1532-4750-a246-2e2881a6be36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-3a48814d-1532-4750-a246-2e2881a6be36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:37.984485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:37.985364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:37.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 140/280] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:37.986827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2efb38-97f9-4a7e-a5dd-35efe9b1fb76 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:37.987143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2efb38-97f9-4a7e-a5dd-35efe9b1fb76 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:37.987457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2efb38-97f9-4a7e-a5dd-35efe9b1fb76 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:37.990618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 141/281] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:37.991614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2efb38-97f9-4a7e-a5dd-35efe9b1fb76 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:38.001257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25996c-8162-4bed-b221-2e942c295d90 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:38.014971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:38.037880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c1ed2-466e-4fda-8837-fcc2a96f6c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:38.038547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c1ed2-466e-4fda-8837-fcc2a96f6c25 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:38.038864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c1ed2-466e-4fda-8837-fcc2a96f6c25 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:38.039148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c1ed2-466e-4fda-8837-fcc2a96f6c25 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:38.039858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c1ed2-466e-4fda-8837-fcc2a96f6c25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:38.041847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c1ed2-466e-4fda-8837-fcc2a96f6c25 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:38.050849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:38.060777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 141/282] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:36:37 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:38.061577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c1ed2-466e-4fda-8837-fcc2a96f6c25 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:38.100291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2efb38-97f9-4a7e-a5dd-35efe9b1fb76 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:38.138989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11d9cf-8db9-44fd-a224-fb0fdd2bae53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:38.139867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11d9cf-8db9-44fd-a224-fb0fdd2bae53 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:38.140149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11d9cf-8db9-44fd-a224-fb0fdd2bae53 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:38.140422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11d9cf-8db9-44fd-a224-fb0fdd2bae53 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:38.141682 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11d9cf-8db9-44fd-a224-fb0fdd2bae53 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:38.143212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11d9cf-8db9-44fd-a224-fb0fdd2bae53 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:38.145188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 142/283] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:37 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:38.148511 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd68170-b720-4b8e-affd-c287969a56de None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:38.149213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd68170-b720-4b8e-affd-c287969a56de None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:38.149834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd68170-b720-4b8e-affd-c287969a56de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:38.168431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-3fd68170-b720-4b8e-affd-c287969a56de None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:38.169659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3fd68170-b720-4b8e-affd-c287969a56de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:38.177108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c1ed2-466e-4fda-8837-fcc2a96f6c25 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:38.215398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3ded2-d545-46f7-b463-904a2938cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:38.215981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3ded2-d545-46f7-b463-904a2938cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:38.216293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3ded2-d545-46f7-b463-904a2938cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:38.216540 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3ded2-d545-46f7-b463-904a2938cfc6 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:38.217303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3ded2-d545-46f7-b463-904a2938cfc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:38.217615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3ded2-d545-46f7-b463-904a2938cfc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:38.219033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3ded2-d545-46f7-b463-904a2938cfc6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:38.227735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 142/284] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:38.229042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3ded2-d545-46f7-b463-904a2938cfc6 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:38.258794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 143/285] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:38.258794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd68170-b720-4b8e-affd-c287969a56de None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:38.275045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:38.327819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:38.378166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd68170-b720-4b8e-affd-c287969a56de None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:38.378675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3ded2-d545-46f7-b463-904a2938cfc6 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:38.420282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd7863-9062-4bd9-81f8-c908e791cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:38.421167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd7863-9062-4bd9-81f8-c908e791cb25 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:38.421457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd7863-9062-4bd9-81f8-c908e791cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:38.421734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd7863-9062-4bd9-81f8-c908e791cb25 None admin] PATH_INFO: `/v3/users/c93efcf1dfc1403484a40a74d3ec0c2a` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:38.426023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe6208-8ecf-46cd-96cc-3d84e2bac1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:38.427173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe6208-8ecf-46cd-96cc-3d84e2bac1a6 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:38.427651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe6208-8ecf-46cd-96cc-3d84e2bac1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:38.428136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe6208-8ecf-46cd-96cc-3d84e2bac1a6 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/users/631d860d924d45b69d717f079ce72c24/roles/aab9a25c2ea047f89513ed57a120fa57` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:38.432073 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe6208-8ecf-46cd-96cc-3d84e2bac1a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fc1137956e4fdb9e9aeb9740c88d9c, user_id=631d860d924d45b69d717f079ce72c24, role_id=aab9a25c2ea047f89513ed57a120fa57)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:38.433789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe6208-8ecf-46cd-96cc-3d84e2bac1a6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:38.441796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd7863-9062-4bd9-81f8-c908e791cb25 None admin] RBAC: Authorizing `identity:get_user(user_id=c93efcf1dfc1403484a40a74d3ec0c2a)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:38.443388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd7863-9062-4bd9-81f8-c908e791cb25 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:38.448632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 144/286] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:38 2022] GET /identity/v3/users/c93efcf1dfc1403484a40a74d3ec0c2a => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:38.449550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd7863-9062-4bd9-81f8-c908e791cb25 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:38.457208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 143/287] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:36:38 2022] PUT /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/users/631d860d924d45b69d717f079ce72c24/roles/aab9a25c2ea047f89513ed57a120fa57 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:38.458393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe6208-8ecf-46cd-96cc-3d84e2bac1a6 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:38.570269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd7863-9062-4bd9-81f8-c908e791cb25 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:38.589750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:38.591531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:38.605741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe6208-8ecf-46cd-96cc-3d84e2bac1a6 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:38.620577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247de83-48c5-4002-98c0-5b125026754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:38.622203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247de83-48c5-4002-98c0-5b125026754c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:38.622960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247de83-48c5-4002-98c0-5b125026754c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:38.623445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247de83-48c5-4002-98c0-5b125026754c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:38.640436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247de83-48c5-4002-98c0-5b125026754c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:38.643571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8547f-b597-4b9c-86b5-8e5711b71352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:38.644333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8547f-b597-4b9c-86b5-8e5711b71352 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:38.644716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8547f-b597-4b9c-86b5-8e5711b71352 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:38.645089 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8547f-b597-4b9c-86b5-8e5711b71352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:38.646026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8547f-b597-4b9c-86b5-8e5711b71352 None admin] RBAC: Adding query filter params (role.id=4d4438a26f504e9f84ba62dc395f926e, user.id=c93efcf1dfc1403484a40a74d3ec0c2a, scope.system=all) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:38.646437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8547f-b597-4b9c-86b5-8e5711b71352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:38.648285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8547f-b597-4b9c-86b5-8e5711b71352 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-d247de83-48c5-4002-98c0-5b125026754c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:38.650970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:38.652073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:38.653085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:38.656472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 145/288] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:38.661181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 144/289] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Jun 15 15:36:38 2022] GET /identity/v3/role_assignments?user.id=c93efcf1dfc1403484a40a74d3ec0c2a&scope.system=all&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:38.662536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247de83-48c5-4002-98c0-5b125026754c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:38.793306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247de83-48c5-4002-98c0-5b125026754c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:38.832444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d91a2-3d25-44da-bdb1-6b396ab3eea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:38.833046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d91a2-3d25-44da-bdb1-6b396ab3eea8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:38.833379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d91a2-3d25-44da-bdb1-6b396ab3eea8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:38.833663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d91a2-3d25-44da-bdb1-6b396ab3eea8 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:38.834430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d91a2-3d25-44da-bdb1-6b396ab3eea8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:38.834755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d91a2-3d25-44da-bdb1-6b396ab3eea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:38.836188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d91a2-3d25-44da-bdb1-6b396ab3eea8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:38.846179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 146/290] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:38.851741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8547f-b597-4b9c-86b5-8e5711b71352 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:38.998685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8547f-b597-4b9c-86b5-8e5711b71352 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:39.050241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8744751-55b6-423f-8158-566a0b9f75b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:39.051154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744751-55b6-423f-8158-566a0b9f75b9 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:39.051438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744751-55b6-423f-8158-566a0b9f75b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:39.051717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744751-55b6-423f-8158-566a0b9f75b9 None admin] PATH_INFO: `/v3/users/c3f06405077e443887484a7f26a43f82` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:39.052937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8744751-55b6-423f-8158-566a0b9f75b9 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f06405077e443887484a7f26a43f82)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:39.054392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8744751-55b6-423f-8158-566a0b9f75b9 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:39.057611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 145/291] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:38 2022] GET /identity/v3/users/c3f06405077e443887484a7f26a43f82 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:39.062296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d91a2-3d25-44da-bdb1-6b396ab3eea8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:39.140924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:39.218508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d91a2-3d25-44da-bdb1-6b396ab3eea8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:39.267389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f37d4-c21f-4016-81bc-6e99b766e172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:39.268040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f37d4-c21f-4016-81bc-6e99b766e172 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:39.268321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f37d4-c21f-4016-81bc-6e99b766e172 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:39.268596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f37d4-c21f-4016-81bc-6e99b766e172 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:39.336557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f37d4-c21f-4016-81bc-6e99b766e172 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:39.338565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f37d4-c21f-4016-81bc-6e99b766e172 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:39.379316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:39.463340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 147/292] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:39.469765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8744751-55b6-423f-8158-566a0b9f75b9 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:39.528083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:39.654588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8744751-55b6-423f-8158-566a0b9f75b9 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:39.704495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492e1f8-6bde-4bec-a57d-8865e37d6458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:39.705314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492e1f8-6bde-4bec-a57d-8865e37d6458 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:39.705604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492e1f8-6bde-4bec-a57d-8865e37d6458 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:39.705889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492e1f8-6bde-4bec-a57d-8865e37d6458 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:39.707195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492e1f8-6bde-4bec-a57d-8865e37d6458 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:39.708748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492e1f8-6bde-4bec-a57d-8865e37d6458 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:39.710828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 146/293] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:39 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:39.716832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f37d4-c21f-4016-81bc-6e99b766e172 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:39.841042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f37d4-c21f-4016-81bc-6e99b766e172 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:39.890409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29497bd8-edf9-4eb3-a3c4-1b10c42a2f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:39.891316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29497bd8-edf9-4eb3-a3c4-1b10c42a2f9d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:39.891657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29497bd8-edf9-4eb3-a3c4-1b10c42a2f9d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:39.892138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29497bd8-edf9-4eb3-a3c4-1b10c42a2f9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:39.893196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29497bd8-edf9-4eb3-a3c4-1b10c42a2f9d None admin] RBAC: Adding query filter params (role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c, user.id=c3f06405077e443887484a7f26a43f82) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:39.893518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29497bd8-edf9-4eb3-a3c4-1b10c42a2f9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:39.895111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29497bd8-edf9-4eb3-a3c4-1b10c42a2f9d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:39.905864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 148/294] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:39 2022] GET /identity/v3/role_assignments?user.id=c3f06405077e443887484a7f26a43f82&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:40.786909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a1fdb-dd89-4d68-9548-fa66d7978124 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:40.787398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a1fdb-dd89-4d68-9548-fa66d7978124 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:40.787398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a1fdb-dd89-4d68-9548-fa66d7978124 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:40.788928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 147/295] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:40.794481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d68b8-3efc-4a8b-a339-f377cbfb3b21 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:40.795342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d68b8-3efc-4a8b-a339-f377cbfb3b21 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:40.795845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d68b8-3efc-4a8b-a339-f377cbfb3b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:40.801664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:40.816037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4c0d68b8-3efc-4a8b-a339-f377cbfb3b21 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:40.817623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4c0d68b8-3efc-4a8b-a339-f377cbfb3b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:40.876296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d89a1-f769-4e48-9394-699f11c30fd1 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:40.876957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d89a1-f769-4e48-9394-699f11c30fd1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:40.877444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d89a1-f769-4e48-9394-699f11c30fd1 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:40.879349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 148/296] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:40.884588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86703319-32dc-4381-af59-3be6b0ae218c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:40.885642 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86703319-32dc-4381-af59-3be6b0ae218c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:40.886123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86703319-32dc-4381-af59-3be6b0ae218c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:40.895539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 149/297] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:40.904056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-86703319-32dc-4381-af59-3be6b0ae218c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:40.905482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-86703319-32dc-4381-af59-3be6b0ae218c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:40.918293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d68b8-3efc-4a8b-a339-f377cbfb3b21 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:40.982273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 149/298] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:41.006645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86703319-32dc-4381-af59-3be6b0ae218c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:41.031154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:41.049443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d68b8-3efc-4a8b-a339-f377cbfb3b21 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:41.084053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:41.090625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81636e1-0ed2-48d4-bf67-a1c00d3c3257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:41.091762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81636e1-0ed2-48d4-bf67-a1c00d3c3257 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.092275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81636e1-0ed2-48d4-bf67-a1c00d3c3257 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.092750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81636e1-0ed2-48d4-bf67-a1c00d3c3257 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.094160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81636e1-0ed2-48d4-bf67-a1c00d3c3257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:41.095958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81636e1-0ed2-48d4-bf67-a1c00d3c3257 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:41.097827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 150/299] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:41.104283 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81636e1-0ed2-48d4-bf67-a1c00d3c3257 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:41.129902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86703319-32dc-4381-af59-3be6b0ae218c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:41.171455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6925d64-8625-4fd3-8a62-68a9f16cae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:41.172630 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6925d64-8625-4fd3-8a62-68a9f16cae59 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.172928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6925d64-8625-4fd3-8a62-68a9f16cae59 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.173207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6925d64-8625-4fd3-8a62-68a9f16cae59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.188282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6925d64-8625-4fd3-8a62-68a9f16cae59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-e6925d64-8625-4fd3-8a62-68a9f16cae59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:41.198475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:41.199449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:41.200260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:41.205223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 150/300] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:41.206601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6925d64-8625-4fd3-8a62-68a9f16cae59 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:41.236319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81636e1-0ed2-48d4-bf67-a1c00d3c3257 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:41.277576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01589ca-544c-4bd3-abb7-61df8ca7d1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:41.278596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01589ca-544c-4bd3-abb7-61df8ca7d1cf None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.279130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01589ca-544c-4bd3-abb7-61df8ca7d1cf None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.279608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01589ca-544c-4bd3-abb7-61df8ca7d1cf None admin] PATH_INFO: `/v3/groups` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.280516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01589ca-544c-4bd3-abb7-61df8ca7d1cf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:41.282170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01589ca-544c-4bd3-abb7-61df8ca7d1cf None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:41.301001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 151/301] 10.222.0.62 () {60 vars in 1134 bytes} [Wed Jun 15 15:36:41 2022] POST /identity/v3/groups => generated 229 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:41.302640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d7f38-7142-41bd-a287-49d0123f49f4 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.303118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d7f38-7142-41bd-a287-49d0123f49f4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.303592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d7f38-7142-41bd-a287-49d0123f49f4 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.305063 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 152/302] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:41.328052 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559569ab-e087-4df6-8eab-441790e10e4b None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.328609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559569ab-e087-4df6-8eab-441790e10e4b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.329058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559569ab-e087-4df6-8eab-441790e10e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.338186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:41.347700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-559569ab-e087-4df6-8eab-441790e10e4b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:41.349152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-559569ab-e087-4df6-8eab-441790e10e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:41.364025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6925d64-8625-4fd3-8a62-68a9f16cae59 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:41.406627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405469d-28c1-43b1-b668-b82cc22acdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:41.407390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405469d-28c1-43b1-b668-b82cc22acdea None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.407679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405469d-28c1-43b1-b668-b82cc22acdea None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.407967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405469d-28c1-43b1-b668-b82cc22acdea None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.408840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405469d-28c1-43b1-b668-b82cc22acdea None admin] RBAC: Adding query filter params (name=reader) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:41.409169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405469d-28c1-43b1-b668-b82cc22acdea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:41.410966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405469d-28c1-43b1-b668-b82cc22acdea None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:41.425341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 151/303] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:41.427817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405469d-28c1-43b1-b668-b82cc22acdea None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:41.431301 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:41.452689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 153/304] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:41.452689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435ce42-9664-4d24-b0c8-b818eb13d061 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.452689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435ce42-9664-4d24-b0c8-b818eb13d061 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.452689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435ce42-9664-4d24-b0c8-b818eb13d061 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.452689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 154/305] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:41.458610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723ff83-0fa9-491d-baea-3d6205f68454 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.458610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723ff83-0fa9-491d-baea-3d6205f68454 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.458847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723ff83-0fa9-491d-baea-3d6205f68454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.476576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-f723ff83-0fa9-491d-baea-3d6205f68454 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:41.477805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f723ff83-0fa9-491d-baea-3d6205f68454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:41.561913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405469d-28c1-43b1-b668-b82cc22acdea None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:41.566946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 155/306] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:41.566946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723ff83-0fa9-491d-baea-3d6205f68454 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:41.601031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a9e76-220a-48c9-afaf-fdb711eeb58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:41.602034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a9e76-220a-48c9-afaf-fdb711eeb58b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.602388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a9e76-220a-48c9-afaf-fdb711eeb58b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.602746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a9e76-220a-48c9-afaf-fdb711eeb58b None admin] PATH_INFO: `/v3/users/9468b6946cc64af18e609fac28b88408` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.615014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:41.618094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a9e76-220a-48c9-afaf-fdb711eeb58b None admin] RBAC: Authorizing `identity:get_user(user_id=9468b6946cc64af18e609fac28b88408)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:41.620492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a9e76-220a-48c9-afaf-fdb711eeb58b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:41.623194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 152/307] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:41 2022] GET /identity/v3/users/9468b6946cc64af18e609fac28b88408 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:41.624381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a9e76-220a-48c9-afaf-fdb711eeb58b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:41.674047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723ff83-0fa9-491d-baea-3d6205f68454 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:41.695686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:41.715718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b17fa-5abc-4f49-a2a0-4e6ae577e17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:41.716519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b17fa-5abc-4f49-a2a0-4e6ae577e17d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.716791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b17fa-5abc-4f49-a2a0-4e6ae577e17d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.717062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b17fa-5abc-4f49-a2a0-4e6ae577e17d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.727932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b17fa-5abc-4f49-a2a0-4e6ae577e17d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-ec5b17fa-5abc-4f49-a2a0-4e6ae577e17d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:41.736128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:41.737029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:41.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:41.743688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a9e76-220a-48c9-afaf-fdb711eeb58b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:41.743774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 156/308] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:41.743774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b17fa-5abc-4f49-a2a0-4e6ae577e17d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:41.778692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2c9f1-d4ea-41ed-842b-a8ddfe9350c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:41.779598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2c9f1-d4ea-41ed-842b-a8ddfe9350c3 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.779956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2c9f1-d4ea-41ed-842b-a8ddfe9350c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.780306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2c9f1-d4ea-41ed-842b-a8ddfe9350c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.791094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2c9f1-d4ea-41ed-842b-a8ddfe9350c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-7cb2c9f1-d4ea-41ed-842b-a8ddfe9350c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:41.798845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:41.799741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:41.800523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 153/309] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:41.801181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2c9f1-d4ea-41ed-842b-a8ddfe9350c3 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:41.857298 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b17fa-5abc-4f49-a2a0-4e6ae577e17d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:41.867805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:41.898137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92f4e9-a22f-4e9c-b0a7-23fd653f94a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:41.899147 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92f4e9-a22f-4e9c-b0a7-23fd653f94a2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.900017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92f4e9-a22f-4e9c-b0a7-23fd653f94a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.900997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92f4e9-a22f-4e9c-b0a7-23fd653f94a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.902261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92f4e9-a22f-4e9c-b0a7-23fd653f94a2 None admin] RBAC: Adding query filter params (role.id=aab9a25c2ea047f89513ed57a120fa57, user.id=9468b6946cc64af18e609fac28b88408, scope.system=all) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:41.902922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92f4e9-a22f-4e9c-b0a7-23fd653f94a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:41.905165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92f4e9-a22f-4e9c-b0a7-23fd653f94a2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:41.911667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 157/310] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Jun 15 15:36:41 2022] GET /identity/v3/role_assignments?user.id=9468b6946cc64af18e609fac28b88408&scope.system=all&role.id=aab9a25c2ea047f89513ed57a120fa57 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:41.912548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2c9f1-d4ea-41ed-842b-a8ddfe9350c3 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:41.912824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92f4e9-a22f-4e9c-b0a7-23fd653f94a2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:41.968287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7915bc1-4658-49a9-b69d-15bbb68fe313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:41.969061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7915bc1-4658-49a9-b69d-15bbb68fe313 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:41.969404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7915bc1-4658-49a9-b69d-15bbb68fe313 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:41.969750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7915bc1-4658-49a9-b69d-15bbb68fe313 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:41.970602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7915bc1-4658-49a9-b69d-15bbb68fe313 None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:41.971023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7915bc1-4658-49a9-b69d-15bbb68fe313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:41.974696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7915bc1-4658-49a9-b69d-15bbb68fe313 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:41.991983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 154/311] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:41.992915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:41.999090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7915bc1-4658-49a9-b69d-15bbb68fe313 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:42.054175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92f4e9-a22f-4e9c-b0a7-23fd653f94a2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:42.093360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4075a-6514-4273-8e57-eb6513617cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:42.094503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4075a-6514-4273-8e57-eb6513617cb5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:42.095116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4075a-6514-4273-8e57-eb6513617cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:42.095590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4075a-6514-4273-8e57-eb6513617cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:42.096537 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4075a-6514-4273-8e57-eb6513617cb5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:42.096852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4075a-6514-4273-8e57-eb6513617cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:42.098229 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4075a-6514-4273-8e57-eb6513617cb5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:42.107847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 158/312] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:42.110523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7915bc1-4658-49a9-b69d-15bbb68fe313 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:42.113169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4075a-6514-4273-8e57-eb6513617cb5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:42.136150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:42.146821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1cc2-86cf-4438-b890-1067f731a373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:42.147972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1cc2-86cf-4438-b890-1067f731a373 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:42.148255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1cc2-86cf-4438-b890-1067f731a373 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:42.148535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1cc2-86cf-4438-b890-1067f731a373 None admin] PATH_INFO: `/v3/users/c93efcf1dfc1403484a40a74d3ec0c2a` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:42.149732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1cc2-86cf-4438-b890-1067f731a373 None admin] RBAC: Authorizing `identity:get_user(user_id=c93efcf1dfc1403484a40a74d3ec0c2a)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:42.151848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1cc2-86cf-4438-b890-1067f731a373 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:42.158273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 155/313] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:41 2022] GET /identity/v3/users/c93efcf1dfc1403484a40a74d3ec0c2a => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:42.159228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1cc2-86cf-4438-b890-1067f731a373 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:42.224488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4075a-6514-4273-8e57-eb6513617cb5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:42.233549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:42.261095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2882e4-052c-4e52-bd43-044d8506e0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:42.262079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2882e4-052c-4e52-bd43-044d8506e0a8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:42.262456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2882e4-052c-4e52-bd43-044d8506e0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:42.262852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2882e4-052c-4e52-bd43-044d8506e0a8 None admin] PATH_INFO: `/v3/users/631d860d924d45b69d717f079ce72c24` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:42.264154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2882e4-052c-4e52-bd43-044d8506e0a8 None admin] RBAC: Authorizing `identity:get_user(user_id=631d860d924d45b69d717f079ce72c24)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:42.265689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2882e4-052c-4e52-bd43-044d8506e0a8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:42.267743 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 159/314] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:42 2022] GET /identity/v3/users/631d860d924d45b69d717f079ce72c24 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:42.274678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2882e4-052c-4e52-bd43-044d8506e0a8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:42.287559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1cc2-86cf-4438-b890-1067f731a373 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:42.325642 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9371331-f9b4-4765-b562-02a4d26ef216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:42.326525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9371331-f9b4-4765-b562-02a4d26ef216 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:42.327536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9371331-f9b4-4765-b562-02a4d26ef216 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:42.327536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9371331-f9b4-4765-b562-02a4d26ef216 None admin] PATH_INFO: `/v3/system/users/c93efcf1dfc1403484a40a74d3ec0c2a/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:42.328618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9371331-f9b4-4765-b562-02a4d26ef216 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c93efcf1dfc1403484a40a74d3ec0c2a, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:42.329993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9371331-f9b4-4765-b562-02a4d26ef216 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:42.340209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 156/315] 10.222.0.62 () {58 vars in 1334 bytes} [Wed Jun 15 15:36:42 2022] PUT /identity/v3/system/users/c93efcf1dfc1403484a40a74d3ec0c2a/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:42.402834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2882e4-052c-4e52-bd43-044d8506e0a8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:42.442978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11cfd9-38c1-4f89-8d46-6ab232dea55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:42.443825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11cfd9-38c1-4f89-8d46-6ab232dea55b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:42.444205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11cfd9-38c1-4f89-8d46-6ab232dea55b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:42.444583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11cfd9-38c1-4f89-8d46-6ab232dea55b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:42.499268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:42.521152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11cfd9-38c1-4f89-8d46-6ab232dea55b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:42.522957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11cfd9-38c1-4f89-8d46-6ab232dea55b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:42.630502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 160/316] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:42.635750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9371331-f9b4-4765-b562-02a4d26ef216 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:42.694413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:42.742296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9371331-f9b4-4765-b562-02a4d26ef216 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:42.776983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a803e3-845e-4d06-99a0-00c4e8118185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:42.777842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a803e3-845e-4d06-99a0-00c4e8118185 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:42.778121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a803e3-845e-4d06-99a0-00c4e8118185 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:42.778394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a803e3-845e-4d06-99a0-00c4e8118185 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:42.779660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a803e3-845e-4d06-99a0-00c4e8118185 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:42.781132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a803e3-845e-4d06-99a0-00c4e8118185 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:42.783042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 157/317] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:42 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:42.788682 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11cfd9-38c1-4f89-8d46-6ab232dea55b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:42.865923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:42.893705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11cfd9-38c1-4f89-8d46-6ab232dea55b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:42.929958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caf927-59f1-4e04-8726-5c4a9701f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:42.930613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caf927-59f1-4e04-8726-5c4a9701f38a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:42.930918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caf927-59f1-4e04-8726-5c4a9701f38a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:42.931192 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caf927-59f1-4e04-8726-5c4a9701f38a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:42.932582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caf927-59f1-4e04-8726-5c4a9701f38a None admin] RBAC: Adding query filter params (role.id=aab9a25c2ea047f89513ed57a120fa57, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c, user.id=631d860d924d45b69d717f079ce72c24) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:42.932891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caf927-59f1-4e04-8726-5c4a9701f38a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:42.934342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caf927-59f1-4e04-8726-5c4a9701f38a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:42.940632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 161/318] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:36:42 2022] GET /identity/v3/role_assignments?user.id=631d860d924d45b69d717f079ce72c24&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=aab9a25c2ea047f89513ed57a120fa57 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:43.544769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490e8eb-47e1-4246-815f-19db7669f93f None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:43.545268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490e8eb-47e1-4246-815f-19db7669f93f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:43.545341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490e8eb-47e1-4246-815f-19db7669f93f None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:43.546855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 158/319] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:43.551953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7e1f3-5e4e-4c3b-99ee-85b302157450 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:43.552246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7e1f3-5e4e-4c3b-99ee-85b302157450 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:43.552726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7e1f3-5e4e-4c3b-99ee-85b302157450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:43.569240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-38f7e1f3-5e4e-4c3b-99ee-85b302157450 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:43.570450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-38f7e1f3-5e4e-4c3b-99ee-85b302157450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:43.639492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 162/320] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:43.662641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490e8eb-47e1-4246-815f-19db7669f93f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:43.766494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490e8eb-47e1-4246-815f-19db7669f93f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:43.788703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:43.801566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c97f-8a21-47fe-9bad-1747e7966559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:43.802360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c97f-8a21-47fe-9bad-1747e7966559 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:43.802644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c97f-8a21-47fe-9bad-1747e7966559 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:43.802950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c97f-8a21-47fe-9bad-1747e7966559 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:43.804106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8c97f-8a21-47fe-9bad-1747e7966559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:43.805535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8c97f-8a21-47fe-9bad-1747e7966559 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:43.807880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 159/321] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:36:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:43.813395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7e1f3-5e4e-4c3b-99ee-85b302157450 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:43.878220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:43.912747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7e1f3-5e4e-4c3b-99ee-85b302157450 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:43.946449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bac36f-9381-4d24-863a-937a7b8359c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:43.947171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bac36f-9381-4d24-863a-937a7b8359c5 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:43.947454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bac36f-9381-4d24-863a-937a7b8359c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:43.947731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bac36f-9381-4d24-863a-937a7b8359c5 None admin] PATH_INFO: `/v3/groups` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:43.948412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bac36f-9381-4d24-863a-937a7b8359c5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:43.949864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bac36f-9381-4d24-863a-937a7b8359c5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:43.968518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 163/322] 10.222.0.62 () {60 vars in 1134 bytes} [Wed Jun 15 15:36:43 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:36:44.096988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cc2e5-66ef-49a0-9c9d-75e497ebb612 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:44.097304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cc2e5-66ef-49a0-9c9d-75e497ebb612 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:44.097466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cc2e5-66ef-49a0-9c9d-75e497ebb612 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:44.099049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 160/323] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:44.104645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022dd6c-ef94-4a51-adab-334b8c3fd843 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:44.104942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022dd6c-ef94-4a51-adab-334b8c3fd843 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:44.105214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022dd6c-ef94-4a51-adab-334b8c3fd843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:44.123461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9022dd6c-ef94-4a51-adab-334b8c3fd843 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:44.124679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9022dd6c-ef94-4a51-adab-334b8c3fd843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:44.201996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 164/324] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:44.223875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cc2e5-66ef-49a0-9c9d-75e497ebb612 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:44.331268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cc2e5-66ef-49a0-9c9d-75e497ebb612 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:44.366685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1c07e-965f-4cf7-be5c-37d200e792b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:44.367828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1c07e-965f-4cf7-be5c-37d200e792b4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:44.368112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1c07e-965f-4cf7-be5c-37d200e792b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:44.368390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1c07e-965f-4cf7-be5c-37d200e792b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:44.379683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1c07e-965f-4cf7-be5c-37d200e792b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-3ea1c07e-965f-4cf7-be5c-37d200e792b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:44.387497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:44.388371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:44.389595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 161/325] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:44.394811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022dd6c-ef94-4a51-adab-334b8c3fd843 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:44.466978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:44.523297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022dd6c-ef94-4a51-adab-334b8c3fd843 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:44.565788 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd73b08-8a01-4855-8f7c-d7d67ba4e4b4 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:44.566290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd73b08-8a01-4855-8f7c-d7d67ba4e4b4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:44.566396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd73b08-8a01-4855-8f7c-d7d67ba4e4b4 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:44.567976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 162/326] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:44.568091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf1bf7-b48b-4f5e-a102-b72a26027818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:44.568763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf1bf7-b48b-4f5e-a102-b72a26027818 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:44.569054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf1bf7-b48b-4f5e-a102-b72a26027818 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:44.569335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf1bf7-b48b-4f5e-a102-b72a26027818 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:44.570100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf1bf7-b48b-4f5e-a102-b72a26027818 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:44.570408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf1bf7-b48b-4f5e-a102-b72a26027818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:44.571974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf1bf7-b48b-4f5e-a102-b72a26027818 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:44.575811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7cf87-0d68-4400-8b6d-a1eb954f4ff7 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:44.576099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7cf87-0d68-4400-8b6d-a1eb954f4ff7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:44.576381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7cf87-0d68-4400-8b6d-a1eb954f4ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:44.582315 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 165/327] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:44.588132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf1bf7-b48b-4f5e-a102-b72a26027818 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:44.608536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-62b7cf87-0d68-4400-8b6d-a1eb954f4ff7 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:44.610334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-62b7cf87-0d68-4400-8b6d-a1eb954f4ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:44.664709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:44.692667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 163/328] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:44.698248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf1bf7-b48b-4f5e-a102-b72a26027818 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:44.712923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7cf87-0d68-4400-8b6d-a1eb954f4ff7 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:44.720080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:44.735784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28476440-a52c-46ee-a592-04df1c08c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:44.736958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28476440-a52c-46ee-a592-04df1c08c073 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:44.736958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28476440-a52c-46ee-a592-04df1c08c073 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:44.737212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28476440-a52c-46ee-a592-04df1c08c073 None admin] PATH_INFO: `/v3/users/9468b6946cc64af18e609fac28b88408` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:44.738500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28476440-a52c-46ee-a592-04df1c08c073 None admin] RBAC: Authorizing `identity:get_user(user_id=9468b6946cc64af18e609fac28b88408)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:44.740230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28476440-a52c-46ee-a592-04df1c08c073 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:44.741952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 166/329] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:44 2022] GET /identity/v3/users/9468b6946cc64af18e609fac28b88408 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:44.747405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28476440-a52c-46ee-a592-04df1c08c073 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:44.828907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7cf87-0d68-4400-8b6d-a1eb954f4ff7 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:44.856313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28476440-a52c-46ee-a592-04df1c08c073 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:44.867188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174f741-136e-4aa3-ac97-18f473a58f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:44.867995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174f741-136e-4aa3-ac97-18f473a58f92 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:44.868280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174f741-136e-4aa3-ac97-18f473a58f92 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:44.868557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174f741-136e-4aa3-ac97-18f473a58f92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:44.880721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174f741-136e-4aa3-ac97-18f473a58f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-d174f741-136e-4aa3-ac97-18f473a58f92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:44.889591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:44.890485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:44.891738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 164/330] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:44.897252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174f741-136e-4aa3-ac97-18f473a58f92 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:44.899124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5b629-4530-418f-bffe-ad4a1d72ec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:44.901234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5b629-4530-418f-bffe-ad4a1d72ec88 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:44.901717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5b629-4530-418f-bffe-ad4a1d72ec88 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:44.902285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5b629-4530-418f-bffe-ad4a1d72ec88 None admin] PATH_INFO: `/v3/system/users/9468b6946cc64af18e609fac28b88408/roles/aab9a25c2ea047f89513ed57a120fa57` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:44.904058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5b629-4530-418f-bffe-ad4a1d72ec88 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9468b6946cc64af18e609fac28b88408, role_id=aab9a25c2ea047f89513ed57a120fa57)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:44.905669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5b629-4530-418f-bffe-ad4a1d72ec88 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:44.919619 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 167/331] 10.222.0.62 () {58 vars in 1334 bytes} [Wed Jun 15 15:36:44 2022] PUT /identity/v3/system/users/9468b6946cc64af18e609fac28b88408/roles/aab9a25c2ea047f89513ed57a120fa57 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:36:44.961726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:45.031539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174f741-136e-4aa3-ac97-18f473a58f92 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:45.075376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd191232-57d4-4298-90e3-ec81a97c8101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:45.076494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd191232-57d4-4298-90e3-ec81a97c8101 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:45.076977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd191232-57d4-4298-90e3-ec81a97c8101 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:45.077451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd191232-57d4-4298-90e3-ec81a97c8101 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:45.078396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd191232-57d4-4298-90e3-ec81a97c8101 None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:45.078936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd191232-57d4-4298-90e3-ec81a97c8101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:45.080505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd191232-57d4-4298-90e3-ec81a97c8101 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:45.090878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 165/332] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:45.096699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5b629-4530-418f-bffe-ad4a1d72ec88 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:45.172306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:45.236498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5b629-4530-418f-bffe-ad4a1d72ec88 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:45.280889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a9ece-6e52-4934-9ab3-b817616a38a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:45.281790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9ece-6e52-4934-9ab3-b817616a38a1 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:45.282078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9ece-6e52-4934-9ab3-b817616a38a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:45.282354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9ece-6e52-4934-9ab3-b817616a38a1 None admin] PATH_INFO: `/v3/users/c93efcf1dfc1403484a40a74d3ec0c2a` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:45.283815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a9ece-6e52-4934-9ab3-b817616a38a1 None admin] RBAC: Authorizing `identity:get_user(user_id=c93efcf1dfc1403484a40a74d3ec0c2a)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:45.285323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a9ece-6e52-4934-9ab3-b817616a38a1 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:45.287222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 168/333] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:45 2022] GET /identity/v3/users/c93efcf1dfc1403484a40a74d3ec0c2a => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:45.293655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd191232-57d4-4298-90e3-ec81a97c8101 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:45.411488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd191232-57d4-4298-90e3-ec81a97c8101 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:45.413088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:45.447729 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1d3c-9c09-495f-9940-dd54ef1b2621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:45.448316 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1d3c-9c09-495f-9940-dd54ef1b2621 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:45.448601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1d3c-9c09-495f-9940-dd54ef1b2621 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:45.448883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1d3c-9c09-495f-9940-dd54ef1b2621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:45.449722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1d3c-9c09-495f-9940-dd54ef1b2621 None admin] RBAC: Adding query filter params (role.id=4d4438a26f504e9f84ba62dc395f926e, user.id=c93efcf1dfc1403484a40a74d3ec0c2a, scope.system=all) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:45.450037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1d3c-9c09-495f-9940-dd54ef1b2621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:45.451972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1d3c-9c09-495f-9940-dd54ef1b2621 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:45.464888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 166/334] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Jun 15 15:36:45 2022] GET /identity/v3/role_assignments?user.id=c93efcf1dfc1403484a40a74d3ec0c2a&scope.system=all&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 510 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:46.535918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0bb1-03ec-42e4-8665-12637f092fbc None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:46.536858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0bb1-03ec-42e4-8665-12637f092fbc None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:46.537423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0bb1-03ec-42e4-8665-12637f092fbc None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:46.539265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 169/335] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:46.545324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7de8a-0a6c-439b-b290-8a4d50069c1f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:46.546154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7de8a-0a6c-439b-b290-8a4d50069c1f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:46.546641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7de8a-0a6c-439b-b290-8a4d50069c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:46.564983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d9f7de8a-0a6c-439b-b290-8a4d50069c1f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:46.566482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d9f7de8a-0a6c-439b-b290-8a4d50069c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:46.647389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 167/336] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:46.680310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef0bb1-03ec-42e4-8665-12637f092fbc None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:46.804253 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef0bb1-03ec-42e4-8665-12637f092fbc None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:46.829045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:46.845254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c5a05-3af4-4d3d-8645-5cb4db0d6007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:46.846072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c5a05-3af4-4d3d-8645-5cb4db0d6007 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:46.846356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c5a05-3af4-4d3d-8645-5cb4db0d6007 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:46.846632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c5a05-3af4-4d3d-8645-5cb4db0d6007 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:46.864201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c5a05-3af4-4d3d-8645-5cb4db0d6007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-4b4c5a05-3af4-4d3d-8645-5cb4db0d6007 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:46.872547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:46.873403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:46.874209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 170/337] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:46.881071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7de8a-0a6c-439b-b290-8a4d50069c1f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:47.100304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d82746-a681-4d6b-aa6e-ccee90c1ca6c None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.100653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d82746-a681-4d6b-aa6e-ccee90c1ca6c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.100825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d82746-a681-4d6b-aa6e-ccee90c1ca6c None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.102437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 171/338] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:47.109612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ab6ed-7d5b-4426-a688-abcb425cdbce None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.110181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ab6ed-7d5b-4426-a688-abcb425cdbce None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.110669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ab6ed-7d5b-4426-a688-abcb425cdbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.117979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:47.129018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7de8a-0a6c-439b-b290-8a4d50069c1f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:47.147769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-871ab6ed-7d5b-4426-a688-abcb425cdbce None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:47.149289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-871ab6ed-7d5b-4426-a688-abcb425cdbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:47.165052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c9279-3a04-42ad-bffe-82c549481970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:47.165760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c9279-3a04-42ad-bffe-82c549481970 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.166046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c9279-3a04-42ad-bffe-82c549481970 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.166326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c9279-3a04-42ad-bffe-82c549481970 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.167121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c9279-3a04-42ad-bffe-82c549481970 None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:47.167431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c9279-3a04-42ad-bffe-82c549481970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:47.168807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c9279-3a04-42ad-bffe-82c549481970 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:47.178378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 168/339] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:47.190338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c9279-3a04-42ad-bffe-82c549481970 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:47.265365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 172/340] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:47.343225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c9279-3a04-42ad-bffe-82c549481970 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:47.364388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ab6ed-7d5b-4426-a688-abcb425cdbce None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:47.386622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251c61c-4ba0-49d1-80e3-16b46388592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:47.387310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251c61c-4ba0-49d1-80e3-16b46388592a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.387593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251c61c-4ba0-49d1-80e3-16b46388592a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.387870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251c61c-4ba0-49d1-80e3-16b46388592a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.412276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:47.502140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251c61c-4ba0-49d1-80e3-16b46388592a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:47.503640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251c61c-4ba0-49d1-80e3-16b46388592a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:47.559747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ab6ed-7d5b-4426-a688-abcb425cdbce None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:47.580278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:47.623493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6fa2b-24df-427d-99f5-154407921990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:47.624430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6fa2b-24df-427d-99f5-154407921990 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.624719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6fa2b-24df-427d-99f5-154407921990 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.624999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6fa2b-24df-427d-99f5-154407921990 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.642774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6fa2b-24df-427d-99f5-154407921990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:47.645799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 169/341] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-30d6fa2b-24df-427d-99f5-154407921990 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:47.648310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:47.649205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:47.650107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 173/342] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:47.651747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251c61c-4ba0-49d1-80e3-16b46388592a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:47.657559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6fa2b-24df-427d-99f5-154407921990 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:47.724492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:47.777945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251c61c-4ba0-49d1-80e3-16b46388592a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:47.813537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d9439-f576-4582-a502-0dcf5866fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:47.814489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d9439-f576-4582-a502-0dcf5866fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.814799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d9439-f576-4582-a502-0dcf5866fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.815082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d9439-f576-4582-a502-0dcf5866fcb4 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.816278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d9439-f576-4582-a502-0dcf5866fcb4 None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:47.817721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d9439-f576-4582-a502-0dcf5866fcb4 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:47.820840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 170/343] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:47 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:47.822190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a998c8-4fc4-47d5-98b5-6f1d7d81e8f1 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.822475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a998c8-4fc4-47d5-98b5-6f1d7d81e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.822786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a998c8-4fc4-47d5-98b5-6f1d7d81e8f1 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.824127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 171/344] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:47.824874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a998c8-4fc4-47d5-98b5-6f1d7d81e8f1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:47.855196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6fa2b-24df-427d-99f5-154407921990 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:47.912453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857c840-2435-4938-a380-3fdeb2241421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:47.913059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857c840-2435-4938-a380-3fdeb2241421 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.913345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857c840-2435-4938-a380-3fdeb2241421 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.913620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857c840-2435-4938-a380-3fdeb2241421 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.914425 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857c840-2435-4938-a380-3fdeb2241421 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:47.914685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857c840-2435-4938-a380-3fdeb2241421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:47.916176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857c840-2435-4938-a380-3fdeb2241421 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:47.931564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a998c8-4fc4-47d5-98b5-6f1d7d81e8f1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:47.933595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 174/345] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:47.934900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ee3c3-0470-438a-97b4-542f1596c759 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.935218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ee3c3-0470-438a-97b4-542f1596c759 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.935470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ee3c3-0470-438a-97b4-542f1596c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.960172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:47.963770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-a38ee3c3-0470-438a-97b4-542f1596c759 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:47.964981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a38ee3c3-0470-438a-97b4-542f1596c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:47.973185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edccc44-0701-409e-b1f9-036b0ea17a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:47.974052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edccc44-0701-409e-b1f9-036b0ea17a74 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:47.974339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edccc44-0701-409e-b1f9-036b0ea17a74 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:47.974614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edccc44-0701-409e-b1f9-036b0ea17a74 None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:47.981476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edccc44-0701-409e-b1f9-036b0ea17a74 None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:47.985074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edccc44-0701-409e-b1f9-036b0ea17a74 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:47.987228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 172/346] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:47 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:47.988138 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edccc44-0701-409e-b1f9-036b0ea17a74 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:48.006190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:48.092405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 175/347] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:48.093287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ee3c3-0470-438a-97b4-542f1596c759 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:48.100979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edccc44-0701-409e-b1f9-036b0ea17a74 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:48.142582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cca178-92d0-4ea7-8ae1-f25bba992619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:48.143682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cca178-92d0-4ea7-8ae1-f25bba992619 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:48.144136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cca178-92d0-4ea7-8ae1-f25bba992619 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:48.144555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cca178-92d0-4ea7-8ae1-f25bba992619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:48.213256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:48.229450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cca178-92d0-4ea7-8ae1-f25bba992619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:48.231178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cca178-92d0-4ea7-8ae1-f25bba992619 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:48.234005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ee3c3-0470-438a-97b4-542f1596c759 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:48.291469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af02b0c-bc0b-43f5-9d01-b1839e0c0771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:48.292150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af02b0c-bc0b-43f5-9d01-b1839e0c0771 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:48.292434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af02b0c-bc0b-43f5-9d01-b1839e0c0771 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:48.292714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af02b0c-bc0b-43f5-9d01-b1839e0c0771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:48.293557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af02b0c-bc0b-43f5-9d01-b1839e0c0771 None admin] RBAC: Adding query filter params (group.id=588218385e8546eab37eb1542ac0d3a9, role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=54d37e7eb5474cf387beec6e56303b4f) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:48.293875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af02b0c-bc0b-43f5-9d01-b1839e0c0771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:48.295369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af02b0c-bc0b-43f5-9d01-b1839e0c0771 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:48.301674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 176/348] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:36:48 2022] GET /identity/v3/role_assignments?group.id=588218385e8546eab37eb1542ac0d3a9&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:48.302537 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af02b0c-bc0b-43f5-9d01-b1839e0c0771 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:48.343294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 173/349] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:48.348896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cca178-92d0-4ea7-8ae1-f25bba992619 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:48.388717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:48.459209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:48.474751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cca178-92d0-4ea7-8ae1-f25bba992619 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:48.523011 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0042ce-1efd-40c5-adaf-f1cd33a5973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:48.524785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0042ce-1efd-40c5-adaf-f1cd33a5973b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:48.525219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0042ce-1efd-40c5-adaf-f1cd33a5973b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:48.525643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0042ce-1efd-40c5-adaf-f1cd33a5973b None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:48.527086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0042ce-1efd-40c5-adaf-f1cd33a5973b None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:48.528878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0042ce-1efd-40c5-adaf-f1cd33a5973b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:48.531286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 174/350] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:48 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:48.539751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0042ce-1efd-40c5-adaf-f1cd33a5973b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:48.665065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af02b0c-bc0b-43f5-9d01-b1839e0c0771 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:48.691193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0042ce-1efd-40c5-adaf-f1cd33a5973b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:48.741245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b7de-4eec-4fde-a9c5-046b97e134eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:48.742253 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b7de-4eec-4fde-a9c5-046b97e134eb None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:48.742632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b7de-4eec-4fde-a9c5-046b97e134eb None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:48.743100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b7de-4eec-4fde-a9c5-046b97e134eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:48.762423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7b7de-4eec-4fde-a9c5-046b97e134eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:48.763041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b34caf-d40f-49d1-ad8b-d74b37660f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:48.764074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34caf-d40f-49d1-ad8b-d74b37660f4b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:48.764498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34caf-d40f-49d1-ad8b-d74b37660f4b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:48.764935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34caf-d40f-49d1-ad8b-d74b37660f4b None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:48.766259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b34caf-d40f-49d1-ad8b-d74b37660f4b None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:48.767884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b34caf-d40f-49d1-ad8b-d74b37660f4b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-adb7b7de-4eec-4fde-a9c5-046b97e134eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:48.772474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:48.773583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:48.775274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:48.776114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 175/351] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:48 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:48.777144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 177/352] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 475 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:48.777492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b34caf-d40f-49d1-ad8b-d74b37660f4b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:48.782445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b7de-4eec-4fde-a9c5-046b97e134eb None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:48.820337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:48.960075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:48.982360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b7de-4eec-4fde-a9c5-046b97e134eb None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:48.996345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b34caf-d40f-49d1-ad8b-d74b37660f4b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:49.033697 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54885b20-2af2-4790-97c3-0fc38aef66ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:49.034741 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54885b20-2af2-4790-97c3-0fc38aef66ef None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.035047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54885b20-2af2-4790-97c3-0fc38aef66ef None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.035326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54885b20-2af2-4790-97c3-0fc38aef66ef None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.036088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54885b20-2af2-4790-97c3-0fc38aef66ef None admin] RBAC: Adding query filter params (name=member) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:49.036491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54885b20-2af2-4790-97c3-0fc38aef66ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:49.037845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4cdfa-392d-427c-804e-2119df8a3c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:49.037971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54885b20-2af2-4790-97c3-0fc38aef66ef None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:49.038512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4cdfa-392d-427c-804e-2119df8a3c04 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.038837 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4cdfa-392d-427c-804e-2119df8a3c04 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.039121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4cdfa-392d-427c-804e-2119df8a3c04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.040078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4cdfa-392d-427c-804e-2119df8a3c04 None admin] RBAC: Adding query filter params (group.id=588218385e8546eab37eb1542ac0d3a9, role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=54d37e7eb5474cf387beec6e56303b4f) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:49.041037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4cdfa-392d-427c-804e-2119df8a3c04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:49.042460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4cdfa-392d-427c-804e-2119df8a3c04 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:49.049106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 178/353] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:49.050494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8060615-cad8-4e05-b163-5cf7dde777e1 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.050843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8060615-cad8-4e05-b163-5cf7dde777e1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.051117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8060615-cad8-4e05-b163-5cf7dde777e1 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.052608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 179/355] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:49.052806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 176/355] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:36:48 2022] GET /identity/v3/role_assignments?group.id=588218385e8546eab37eb1542ac0d3a9&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 260 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:49.053976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80359aa6-0e89-47d8-a4b7-73a14bf8b9fc None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.054258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80359aa6-0e89-47d8-a4b7-73a14bf8b9fc None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.054530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80359aa6-0e89-47d8-a4b7-73a14bf8b9fc None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.054980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4cdfa-392d-427c-804e-2119df8a3c04 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:49.056548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 180/356] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:49.062234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcc7f-ed23-4fa8-b99a-e1d42df10b08 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.062524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcc7f-ed23-4fa8-b99a-e1d42df10b08 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.062809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcc7f-ed23-4fa8-b99a-e1d42df10b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.102114 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b71fcc7f-ed23-4fa8-b99a-e1d42df10b08 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:49.103938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b71fcc7f-ed23-4fa8-b99a-e1d42df10b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:49.186432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:49.219702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4cdfa-392d-427c-804e-2119df8a3c04 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:49.257850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 181/357] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:49.260152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3319d5-32f6-4f06-914e-023ccea10430 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.260678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3319d5-32f6-4f06-914e-023ccea10430 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.261156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3319d5-32f6-4f06-914e-023ccea10430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.279418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-7c3319d5-32f6-4f06-914e-023ccea10430 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:49.281152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7c3319d5-32f6-4f06-914e-023ccea10430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:49.285596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fccc5-d246-423d-ba70-8a15631ce4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:49.286268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fccc5-d246-423d-ba70-8a15631ce4f3 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.286555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fccc5-d246-423d-ba70-8a15631ce4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.286881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fccc5-d246-423d-ba70-8a15631ce4f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.378299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750fccc5-d246-423d-ba70-8a15631ce4f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:49.378835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 182/358] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:49.380164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386ff87-c1db-4c79-9fa1-da367304d4e2 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.380514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386ff87-c1db-4c79-9fa1-da367304d4e2 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.380857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386ff87-c1db-4c79-9fa1-da367304d4e2 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.381186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750fccc5-d246-423d-ba70-8a15631ce4f3 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:49.382258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 183/359] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:49.386260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386ff87-c1db-4c79-9fa1-da367304d4e2 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:49.395578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:49.520322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:49.531791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386ff87-c1db-4c79-9fa1-da367304d4e2 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:49.541887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 177/360] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:49.543613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9736c-eccb-4b1c-8b61-6eb9e8b6a62f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.544093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9736c-eccb-4b1c-8b61-6eb9e8b6a62f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.544552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9736c-eccb-4b1c-8b61-6eb9e8b6a62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.562449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-11b9736c-eccb-4b1c-8b61-6eb9e8b6a62f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:49.563982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-11b9736c-eccb-4b1c-8b61-6eb9e8b6a62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:49.576481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b92313-05ec-40cd-a264-37f3c5ba0578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:49.577565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b92313-05ec-40cd-a264-37f3c5ba0578 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.577676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b92313-05ec-40cd-a264-37f3c5ba0578 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.578099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b92313-05ec-40cd-a264-37f3c5ba0578 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.590954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b92313-05ec-40cd-a264-37f3c5ba0578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-35b92313-05ec-40cd-a264-37f3c5ba0578 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:49.599472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:49.601142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 184/361] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:49.601963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b92313-05ec-40cd-a264-37f3c5ba0578 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:49.664051 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 178/362] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:49.664797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9736c-eccb-4b1c-8b61-6eb9e8b6a62f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:49.704468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:49.750370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b92313-05ec-40cd-a264-37f3c5ba0578 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:49.797575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b19852-44f2-42b2-b903-06cfaf9eaf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:49.798696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b19852-44f2-42b2-b903-06cfaf9eaf19 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.799269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b19852-44f2-42b2-b903-06cfaf9eaf19 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.799767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b19852-44f2-42b2-b903-06cfaf9eaf19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.807421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9736c-eccb-4b1c-8b61-6eb9e8b6a62f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:49.815438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b19852-44f2-42b2-b903-06cfaf9eaf19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:49.823649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-30b19852-44f2-42b2-b903-06cfaf9eaf19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:49.824715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:49.825632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 15:36:49.826510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:49.829360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 185/363] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:49.830396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b19852-44f2-42b2-b903-06cfaf9eaf19 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:49.849252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760244-4cc5-4d6c-bcf3-3b03c376dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:49.850403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760244-4cc5-4d6c-bcf3-3b03c376dda3 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.850693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760244-4cc5-4d6c-bcf3-3b03c376dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.851009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760244-4cc5-4d6c-bcf3-3b03c376dda3 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.852376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760244-4cc5-4d6c-bcf3-3b03c376dda3 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:49.854405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760244-4cc5-4d6c-bcf3-3b03c376dda3 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:49.855921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 179/364] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:49 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:49.856685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760244-4cc5-4d6c-bcf3-3b03c376dda3 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:49.953606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b19852-44f2-42b2-b903-06cfaf9eaf19 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:49.984267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:49.988494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760244-4cc5-4d6c-bcf3-3b03c376dda3 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:49.996058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d551f1-c197-488e-b806-3f735f8a939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:49.997079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d551f1-c197-488e-b806-3f735f8a939a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:49.997566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d551f1-c197-488e-b806-3f735f8a939a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:49.998104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d551f1-c197-488e-b806-3f735f8a939a None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:49.999142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d551f1-c197-488e-b806-3f735f8a939a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:49.999670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d551f1-c197-488e-b806-3f735f8a939a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:50.001292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d551f1-c197-488e-b806-3f735f8a939a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:50.011498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 186/365] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:50.012604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d551f1-c197-488e-b806-3f735f8a939a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:50.029570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c812d16-a1a9-427b-b379-60d530d24f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:50.030866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c812d16-a1a9-427b-b379-60d530d24f5f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:50.031390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c812d16-a1a9-427b-b379-60d530d24f5f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:50.031951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c812d16-a1a9-427b-b379-60d530d24f5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:50.045820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c812d16-a1a9-427b-b379-60d530d24f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-1c812d16-a1a9-427b-b379-60d530d24f5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:50.062821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:50.064592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:50.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:50.069033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 180/366] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:50.070095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c812d16-a1a9-427b-b379-60d530d24f5f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:50.126698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:50.148287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d551f1-c197-488e-b806-3f735f8a939a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:50.194780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd8b37-8afa-4b26-833d-c9613744ac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:50.195207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd8b37-8afa-4b26-833d-c9613744ac91 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:50.195496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd8b37-8afa-4b26-833d-c9613744ac91 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:50.195781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd8b37-8afa-4b26-833d-c9613744ac91 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:50.197658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd8b37-8afa-4b26-833d-c9613744ac91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:50.198104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd8b37-8afa-4b26-833d-c9613744ac91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:50.199533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd8b37-8afa-4b26-833d-c9613744ac91 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:50.199765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c812d16-a1a9-427b-b379-60d530d24f5f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:50.214619 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 187/367] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:50.216502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd8b37-8afa-4b26-833d-c9613744ac91 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:50.247312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a23e99-74f8-4666-bc14-97e772478439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:50.248430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a23e99-74f8-4666-bc14-97e772478439 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:50.248723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a23e99-74f8-4666-bc14-97e772478439 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:50.249009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a23e99-74f8-4666-bc14-97e772478439 None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:50.250640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a23e99-74f8-4666-bc14-97e772478439 None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:50.252658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a23e99-74f8-4666-bc14-97e772478439 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:50.255059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 181/368] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:50 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:50.255835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a23e99-74f8-4666-bc14-97e772478439 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:50.281255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:50.340409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd8b37-8afa-4b26-833d-c9613744ac91 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:50.377472 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a23e99-74f8-4666-bc14-97e772478439 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:50.379884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cf1b7-91e8-477c-9a75-e198e303ba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:50.380778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cf1b7-91e8-477c-9a75-e198e303ba16 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:50.382202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cf1b7-91e8-477c-9a75-e198e303ba16 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:50.382695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cf1b7-91e8-477c-9a75-e198e303ba16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:50.389762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:50.417856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586f67-1e47-47cf-90e9-5521ec08a252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:50.418555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586f67-1e47-47cf-90e9-5521ec08a252 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:50.418883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586f67-1e47-47cf-90e9-5521ec08a252 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:50.419164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586f67-1e47-47cf-90e9-5521ec08a252 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:50.419923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586f67-1e47-47cf-90e9-5521ec08a252 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:50.420232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586f67-1e47-47cf-90e9-5521ec08a252 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:50.421602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586f67-1e47-47cf-90e9-5521ec08a252 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:50.434107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 182/369] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:50.434107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586f67-1e47-47cf-90e9-5521ec08a252 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:50.477127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cf1b7-91e8-477c-9a75-e198e303ba16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:50.479573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cf1b7-91e8-477c-9a75-e198e303ba16 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:50.546258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586f67-1e47-47cf-90e9-5521ec08a252 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:50.559560 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:50.584995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe79c7-6b8c-40ae-8f82-6dd1087f5020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:50.586041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe79c7-6b8c-40ae-8f82-6dd1087f5020 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:50.586425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe79c7-6b8c-40ae-8f82-6dd1087f5020 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:50.586831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe79c7-6b8c-40ae-8f82-6dd1087f5020 None admin] PATH_INFO: `/v3/users/9468b6946cc64af18e609fac28b88408` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:50.588663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe79c7-6b8c-40ae-8f82-6dd1087f5020 None admin] RBAC: Authorizing `identity:get_user(user_id=9468b6946cc64af18e609fac28b88408)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:50.590202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe79c7-6b8c-40ae-8f82-6dd1087f5020 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:50.592506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 183/370] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:36:50 2022] GET /identity/v3/users/9468b6946cc64af18e609fac28b88408 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:50.593210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe79c7-6b8c-40ae-8f82-6dd1087f5020 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:50.601010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 188/371] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:50.602160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cf1b7-91e8-477c-9a75-e198e303ba16 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:50.641981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:50.707672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe79c7-6b8c-40ae-8f82-6dd1087f5020 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:50.718709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cf1b7-91e8-477c-9a75-e198e303ba16 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:50.747912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1ef6f-c213-40cd-9897-cd956ebcd471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:50.748466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1ef6f-c213-40cd-9897-cd956ebcd471 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:50.748825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1ef6f-c213-40cd-9897-cd956ebcd471 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:50.749178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1ef6f-c213-40cd-9897-cd956ebcd471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:50.750315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1ef6f-c213-40cd-9897-cd956ebcd471 None admin] RBAC: Adding query filter params (group.id=588218385e8546eab37eb1542ac0d3a9, role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:50.750700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1ef6f-c213-40cd-9897-cd956ebcd471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:50.752232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1ef6f-c213-40cd-9897-cd956ebcd471 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:50.760898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237606a7-73aa-46ed-9c26-a78fc39c2901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:50.761837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237606a7-73aa-46ed-9c26-a78fc39c2901 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:50.762452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237606a7-73aa-46ed-9c26-a78fc39c2901 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:50.762935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237606a7-73aa-46ed-9c26-a78fc39c2901 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:50.763168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 184/372] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:36:50 2022] GET /identity/v3/role_assignments?group.id=588218385e8546eab37eb1542ac0d3a9&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:50.764172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1ef6f-c213-40cd-9897-cd956ebcd471 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:50.815262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:50.905755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1ef6f-c213-40cd-9897-cd956ebcd471 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:50.908905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:50.939015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237606a7-73aa-46ed-9c26-a78fc39c2901 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:50.940640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237606a7-73aa-46ed-9c26-a78fc39c2901 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:50.946446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee7530-99e8-4c2d-a1a3-a5894888eea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:50.947730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee7530-99e8-4c2d-a1a3-a5894888eea6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:50.948023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee7530-99e8-4c2d-a1a3-a5894888eea6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:50.948308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee7530-99e8-4c2d-a1a3-a5894888eea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:50.949165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee7530-99e8-4c2d-a1a3-a5894888eea6 None admin] RBAC: Adding query filter params (role.id=aab9a25c2ea047f89513ed57a120fa57, user.id=9468b6946cc64af18e609fac28b88408, scope.system=all) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:50.949485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee7530-99e8-4c2d-a1a3-a5894888eea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:50.950981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee7530-99e8-4c2d-a1a3-a5894888eea6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:50.965425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 185/373] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Jun 15 15:36:50 2022] GET /identity/v3/role_assignments?user.id=9468b6946cc64af18e609fac28b88408&scope.system=all&role.id=aab9a25c2ea047f89513ed57a120fa57 => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:50.965627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee7530-99e8-4c2d-a1a3-a5894888eea6 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:51.078939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee7530-99e8-4c2d-a1a3-a5894888eea6 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:51.080430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 189/374] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:51.086581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237606a7-73aa-46ed-9c26-a78fc39c2901 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:51.118943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee9645-1e07-4304-bd48-9d21d189d0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:51.119814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee9645-1e07-4304-bd48-9d21d189d0cd None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.120100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee9645-1e07-4304-bd48-9d21d189d0cd None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.120377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee9645-1e07-4304-bd48-9d21d189d0cd None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.122186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee9645-1e07-4304-bd48-9d21d189d0cd None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:51.123758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee9645-1e07-4304-bd48-9d21d189d0cd None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:51.125698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 186/375] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:50 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:51.131537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee9645-1e07-4304-bd48-9d21d189d0cd None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:51.168897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:51.183172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:51.247146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237606a7-73aa-46ed-9c26-a78fc39c2901 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:51.269936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee9645-1e07-4304-bd48-9d21d189d0cd None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:51.287306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8b1fa-502f-427b-83a9-10f717f88d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:51.288197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8b1fa-502f-427b-83a9-10f717f88d2c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.288484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8b1fa-502f-427b-83a9-10f717f88d2c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.288763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8b1fa-502f-427b-83a9-10f717f88d2c None admin] PATH_INFO: `/v3/projects/87109ec03f124710b1afe403b1e173c0` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.289994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8b1fa-502f-427b-83a9-10f717f88d2c None admin] RBAC: Authorizing `identity:get_project(project_id=87109ec03f124710b1afe403b1e173c0)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:51.292034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8b1fa-502f-427b-83a9-10f717f88d2c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:51.294187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 190/376] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:51 2022] GET /identity/v3/projects/87109ec03f124710b1afe403b1e173c0 => generated 344 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:51.300264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8b1fa-502f-427b-83a9-10f717f88d2c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:51.313492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb0b61-b15a-4905-9bc3-425e3f2227a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:51.314690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0b61-b15a-4905-9bc3-425e3f2227a5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.315295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0b61-b15a-4905-9bc3-425e3f2227a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.315586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0b61-b15a-4905-9bc3-425e3f2227a5 None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.319526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb0b61-b15a-4905-9bc3-425e3f2227a5 None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:51.321003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb0b61-b15a-4905-9bc3-425e3f2227a5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:51.325047 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 187/377] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:51 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:51.331210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb0b61-b15a-4905-9bc3-425e3f2227a5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:51.434810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8b1fa-502f-427b-83a9-10f717f88d2c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:51.462044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:51.465121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:51.470772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb0b61-b15a-4905-9bc3-425e3f2227a5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:51.487878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01be3f-04d1-4596-a51c-227a55a7f819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:51.489111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01be3f-04d1-4596-a51c-227a55a7f819 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.489677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01be3f-04d1-4596-a51c-227a55a7f819 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.489968 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01be3f-04d1-4596-a51c-227a55a7f819 None admin] PATH_INFO: `/v3/groups/dfb27ed5df9c4aa7915a854083d47857` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.499817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01be3f-04d1-4596-a51c-227a55a7f819 None admin] RBAC: Authorizing `identity:get_group(group_id=dfb27ed5df9c4aa7915a854083d47857)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:51.501797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01be3f-04d1-4596-a51c-227a55a7f819 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:51.504917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 191/378] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:51 2022] GET /identity/v3/groups/dfb27ed5df9c4aa7915a854083d47857 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:51.506577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f2c22-5fa0-482d-a7be-8b05698ed42f None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.506953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f2c22-5fa0-482d-a7be-8b05698ed42f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.507238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f2c22-5fa0-482d-a7be-8b05698ed42f None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.509570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 192/379] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:51.512385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f2c22-5fa0-482d-a7be-8b05698ed42f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:51.527513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea92ee-24a2-4b4b-995c-73e16d33fffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:51.528646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea92ee-24a2-4b4b-995c-73e16d33fffe None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.528939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea92ee-24a2-4b4b-995c-73e16d33fffe None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.529222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea92ee-24a2-4b4b-995c-73e16d33fffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.530503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea92ee-24a2-4b4b-995c-73e16d33fffe None admin] RBAC: Adding query filter params (group.id=588218385e8546eab37eb1542ac0d3a9, role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:51.531158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea92ee-24a2-4b4b-995c-73e16d33fffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:51.532954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea92ee-24a2-4b4b-995c-73e16d33fffe None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:51.541588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 188/380] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:36:51 2022] GET /identity/v3/role_assignments?group.id=588218385e8546eab37eb1542ac0d3a9&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:51.542924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d292ee-32a5-4483-a924-b75b09bafd50 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.543220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d292ee-32a5-4483-a924-b75b09bafd50 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.543501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d292ee-32a5-4483-a924-b75b09bafd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.568421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-83d292ee-32a5-4483-a924-b75b09bafd50 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:51.569663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-83d292ee-32a5-4483-a924-b75b09bafd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:51.701985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 189/381] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:51.718690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f2c22-5fa0-482d-a7be-8b05698ed42f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:51.737121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d292ee-32a5-4483-a924-b75b09bafd50 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:51.788631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33c033-dc84-403e-80d0-53bf706c8c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:51.789306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33c033-dc84-403e-80d0-53bf706c8c4c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.789588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33c033-dc84-403e-80d0-53bf706c8c4c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.789867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33c033-dc84-403e-80d0-53bf706c8c4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.790775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33c033-dc84-403e-80d0-53bf706c8c4c None admin] RBAC: Adding query filter params (group.id=dfb27ed5df9c4aa7915a854083d47857, role.id=433478cc318a467c931f13a757f161fa, scope.project.id=87109ec03f124710b1afe403b1e173c0) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:51.791100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33c033-dc84-403e-80d0-53bf706c8c4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:51.794376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33c033-dc84-403e-80d0-53bf706c8c4c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:51.803826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 193/382] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:36:51 2022] GET /identity/v3/role_assignments?group.id=dfb27ed5df9c4aa7915a854083d47857&scope.project.id=87109ec03f124710b1afe403b1e173c0&role.id=433478cc318a467c931f13a757f161fa => generated 260 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:51.835583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d948-4c4b-4b4d-a660-2700d01019f7 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.835906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d948-4c4b-4b4d-a660-2700d01019f7 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.836145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d948-4c4b-4b4d-a660-2700d01019f7 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.838343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 194/383] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:36:51.844785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5fb24-0b07-4b6c-a402-33f91265ab54 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.844785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5fb24-0b07-4b6c-a402-33f91265ab54 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.844785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5fb24-0b07-4b6c-a402-33f91265ab54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.866887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-28f5fb24-0b07-4b6c-a402-33f91265ab54 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:51.868825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-28f5fb24-0b07-4b6c-a402-33f91265ab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:51.911419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:51.925203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d292ee-32a5-4483-a924-b75b09bafd50 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:51.938293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:51.969259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7b515-7959-4791-a248-b8db835d53ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:51.970414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7b515-7959-4791-a248-b8db835d53ea None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:51.970942 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7b515-7959-4791-a248-b8db835d53ea None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:51.971447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7b515-7959-4791-a248-b8db835d53ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:51.985550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7b515-7959-4791-a248-b8db835d53ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-64d7b515-7959-4791-a248-b8db835d53ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:51.996234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:51.997166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:51.998002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:51.998650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 195/384] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:52.000188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 190/385] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:52.016734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5fb24-0b07-4b6c-a402-33f91265ab54 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:52.031553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7b515-7959-4791-a248-b8db835d53ea None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:52.205330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5fb24-0b07-4b6c-a402-33f91265ab54 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:52.224979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7b515-7959-4791-a248-b8db835d53ea None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:52.274377 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4bca8-7964-4420-9fb8-c36ba08cc875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:52.276465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4bca8-7964-4420-9fb8-c36ba08cc875 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:52.277650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4bca8-7964-4420-9fb8-c36ba08cc875 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:52.278139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4bca8-7964-4420-9fb8-c36ba08cc875 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:52.279734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4bca8-7964-4420-9fb8-c36ba08cc875 None admin] RBAC: Adding query filter params (name=member) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:52.280420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4bca8-7964-4420-9fb8-c36ba08cc875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:52.283082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4bca8-7964-4420-9fb8-c36ba08cc875 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:52.294332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e1283-aba6-4a91-8754-d5eae3ddd479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:52.295560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e1283-aba6-4a91-8754-d5eae3ddd479 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:52.296082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e1283-aba6-4a91-8754-d5eae3ddd479 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:52.297687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e1283-aba6-4a91-8754-d5eae3ddd479 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:52.303183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 196/386] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:52.311636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4bca8-7964-4420-9fb8-c36ba08cc875 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:52.326757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e1283-aba6-4a91-8754-d5eae3ddd479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-063e1283-aba6-4a91-8754-d5eae3ddd479 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:52.341805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:52.342797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:52.343608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:52.348347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 191/387] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:52.348862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e1283-aba6-4a91-8754-d5eae3ddd479 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:52.380808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:52.386968 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:52.520823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4bca8-7964-4420-9fb8-c36ba08cc875 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:52.531177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e1283-aba6-4a91-8754-d5eae3ddd479 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:52.567668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eca43c-1758-4a4f-9dfe-1b044a094300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:52.568344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eca43c-1758-4a4f-9dfe-1b044a094300 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:52.568737 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eca43c-1758-4a4f-9dfe-1b044a094300 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:52.569029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eca43c-1758-4a4f-9dfe-1b044a094300 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:52.569787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eca43c-1758-4a4f-9dfe-1b044a094300 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:52.570102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eca43c-1758-4a4f-9dfe-1b044a094300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:52.571567 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eca43c-1758-4a4f-9dfe-1b044a094300 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:52.581588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 192/388] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:52.582900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741dea93-f4e1-4262-bbda-8c3853a32225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:52.583882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dea93-f4e1-4262-bbda-8c3853a32225 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:52.584166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dea93-f4e1-4262-bbda-8c3853a32225 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:52.584441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dea93-f4e1-4262-bbda-8c3853a32225 None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:52.585680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741dea93-f4e1-4262-bbda-8c3853a32225 None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:52.587501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741dea93-f4e1-4262-bbda-8c3853a32225 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:52.588410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eca43c-1758-4a4f-9dfe-1b044a094300 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:52.589504 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 197/389] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:52 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:52.612948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741dea93-f4e1-4262-bbda-8c3853a32225 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:52.676985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:52.699165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eca43c-1758-4a4f-9dfe-1b044a094300 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:52.736540 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b6166-8e6f-4462-8141-044b0f930a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:52.737501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b6166-8e6f-4462-8141-044b0f930a2d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:52.737854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b6166-8e6f-4462-8141-044b0f930a2d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:52.738198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b6166-8e6f-4462-8141-044b0f930a2d None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:52.739625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b6166-8e6f-4462-8141-044b0f930a2d None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:52.741206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b6166-8e6f-4462-8141-044b0f930a2d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:52.745548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 193/390] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:52 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:52.750857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b6166-8e6f-4462-8141-044b0f930a2d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:52.782901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741dea93-f4e1-4262-bbda-8c3853a32225 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:52.797636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:52.838689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241119a-ef8a-4bf3-8422-5be842f804bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:52.839587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241119a-ef8a-4bf3-8422-5be842f804bc None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:52.839874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241119a-ef8a-4bf3-8422-5be842f804bc None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:52.840155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241119a-ef8a-4bf3-8422-5be842f804bc None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:52.841343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241119a-ef8a-4bf3-8422-5be842f804bc None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:52.842813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241119a-ef8a-4bf3-8422-5be842f804bc None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:52.844807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 198/391] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:52 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:52.868021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b6166-8e6f-4462-8141-044b0f930a2d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:52.871736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241119a-ef8a-4bf3-8422-5be842f804bc None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:52.904577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df76b1-ba1b-48d9-8087-059444aff850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:52.905559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df76b1-ba1b-48d9-8087-059444aff850 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:52.905902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df76b1-ba1b-48d9-8087-059444aff850 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:52.906212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df76b1-ba1b-48d9-8087-059444aff850 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:52.907571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df76b1-ba1b-48d9-8087-059444aff850 None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:52.909009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df76b1-ba1b-48d9-8087-059444aff850 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:52.910886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 194/392] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:52 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:52.923049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df76b1-ba1b-48d9-8087-059444aff850 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:52.943122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:53.028578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241119a-ef8a-4bf3-8422-5be842f804bc None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:53.088144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648819df-33ae-4c14-8103-dbd73ea7d0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:53.088967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648819df-33ae-4c14-8103-dbd73ea7d0e4 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:53.089249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648819df-33ae-4c14-8103-dbd73ea7d0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:53.089525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648819df-33ae-4c14-8103-dbd73ea7d0e4 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f/groups/588218385e8546eab37eb1542ac0d3a9/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:53.093914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648819df-33ae-4c14-8103-dbd73ea7d0e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=54d37e7eb5474cf387beec6e56303b4f, group_id=588218385e8546eab37eb1542ac0d3a9, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:53.095498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648819df-33ae-4c14-8103-dbd73ea7d0e4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:53.102047 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df76b1-ba1b-48d9-8087-059444aff850 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:53.122856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 199/393] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Jun 15 15:36:52 2022] PUT /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f/groups/588218385e8546eab37eb1542ac0d3a9/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:53.151702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277b98f-77f0-443c-a7bd-11d824f506e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:53.152625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277b98f-77f0-443c-a7bd-11d824f506e6 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:53.152910 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277b98f-77f0-443c-a7bd-11d824f506e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:53.153332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277b98f-77f0-443c-a7bd-11d824f506e6 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f/groups/588218385e8546eab37eb1542ac0d3a9/roles/7fb980f85eb04293a0d1dba515bf0a2c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:53.155767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277b98f-77f0-443c-a7bd-11d824f506e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=54d37e7eb5474cf387beec6e56303b4f, group_id=588218385e8546eab37eb1542ac0d3a9, role_id=7fb980f85eb04293a0d1dba515bf0a2c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:53.157395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277b98f-77f0-443c-a7bd-11d824f506e6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:53.183131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 195/394] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Jun 15 15:36:52 2022] PUT /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f/groups/588218385e8546eab37eb1542ac0d3a9/roles/7fb980f85eb04293a0d1dba515bf0a2c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:53.703841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ad431-4f9b-4eea-812f-1d3c4643783b None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:53.704026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ad431-4f9b-4eea-812f-1d3c4643783b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:53.704311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ad431-4f9b-4eea-812f-1d3c4643783b None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:53.705703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 200/395] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:53.711186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094a5b-af99-461c-b920-0325b5742ea2 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:53.711482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094a5b-af99-461c-b920-0325b5742ea2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:53.712304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094a5b-af99-461c-b920-0325b5742ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:53.733759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-bc094a5b-af99-461c-b920-0325b5742ea2 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:53.734971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-bc094a5b-af99-461c-b920-0325b5742ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:53.822722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:53.824990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 196/396] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:53.851752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ad431-4f9b-4eea-812f-1d3c4643783b None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:53.907837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:53.992378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ad431-4f9b-4eea-812f-1d3c4643783b None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:54.031612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a0124-eb51-4461-9921-a04956d881aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:54.032414 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a0124-eb51-4461-9921-a04956d881aa None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.032745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a0124-eb51-4461-9921-a04956d881aa None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.033026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a0124-eb51-4461-9921-a04956d881aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:54.046076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a0124-eb51-4461-9921-a04956d881aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-f25a0124-eb51-4461-9921-a04956d881aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:54.054019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:54.054940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:54.055790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 201/397] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:54.062999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc094a5b-af99-461c-b920-0325b5742ea2 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:54.182210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc094a5b-af99-461c-b920-0325b5742ea2 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:54.223616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5828-9063-44b1-8913-8666adfbf9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:54.224700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5828-9063-44b1-8913-8666adfbf9e1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.225187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5828-9063-44b1-8913-8666adfbf9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.225665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5828-9063-44b1-8913-8666adfbf9e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:54.226627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea5828-9063-44b1-8913-8666adfbf9e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:54.227172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea5828-9063-44b1-8913-8666adfbf9e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:54.228797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea5828-9063-44b1-8913-8666adfbf9e1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:54.242713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 197/398] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:54.248559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a0124-eb51-4461-9921-a04956d881aa None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:54.392762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:54.399150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a0124-eb51-4461-9921-a04956d881aa None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:54.444154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0700d-426f-4c4a-a85a-d8ae2ce59dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:54.447532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0700d-426f-4c4a-a85a-d8ae2ce59dc0 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.447532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0700d-426f-4c4a-a85a-d8ae2ce59dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.447532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0700d-426f-4c4a-a85a-d8ae2ce59dc0 None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:54.447532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0700d-426f-4c4a-a85a-d8ae2ce59dc0 None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:54.448310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0700d-426f-4c4a-a85a-d8ae2ce59dc0 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:54.450269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 202/399] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:54 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:54.455467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5828-9063-44b1-8913-8666adfbf9e1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:54.543600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:54.554985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26493ee8-7cec-463d-ab06-b843ac6850ec None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.555334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26493ee8-7cec-463d-ab06-b843ac6850ec None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.556077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26493ee8-7cec-463d-ab06-b843ac6850ec None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:54.557548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 203/400] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:54.563679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920ac05-0da0-49fd-ac17-6f86d4601bd4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.564183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920ac05-0da0-49fd-ac17-6f86d4601bd4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.564468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920ac05-0da0-49fd-ac17-6f86d4601bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:54.595414 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-a920ac05-0da0-49fd-ac17-6f86d4601bd4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:54.596608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a920ac05-0da0-49fd-ac17-6f86d4601bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:54.624006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5828-9063-44b1-8913-8666adfbf9e1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:54.667878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d0c6a-28af-4631-bee3-63d98eb3c0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:54.669010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d0c6a-28af-4631-bee3-63d98eb3c0e4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.669293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d0c6a-28af-4631-bee3-63d98eb3c0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.669827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d0c6a-28af-4631-bee3-63d98eb3c0e4 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:54.671539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d0c6a-28af-4631-bee3-63d98eb3c0e4 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:54.672983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d0c6a-28af-4631-bee3-63d98eb3c0e4 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:54.675177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 198/401] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:54 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:54.677085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e152c64-9b05-407e-a468-320a565bc3df None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.677640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e152c64-9b05-407e-a468-320a565bc3df None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.677933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e152c64-9b05-407e-a468-320a565bc3df None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:54.679665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 199/402] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:54.681278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e152c64-9b05-407e-a468-320a565bc3df None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:54.690419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 204/403] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:54.691902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b803-06f4-46b3-b105-66484b1011d5 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.692194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b803-06f4-46b3-b105-66484b1011d5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.692471 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b803-06f4-46b3-b105-66484b1011d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:54.722931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-a948b803-06f4-46b3-b105-66484b1011d5 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:54.724197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a948b803-06f4-46b3-b105-66484b1011d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:54.807262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 205/404] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:54.808074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948b803-06f4-46b3-b105-66484b1011d5 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:54.810689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e152c64-9b05-407e-a468-320a565bc3df None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:54.842339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:54.848120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a745-feeb-4df5-93a8-a2659ea45160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:54.849002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a745-feeb-4df5-93a8-a2659ea45160 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.849281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a745-feeb-4df5-93a8-a2659ea45160 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.849672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a745-feeb-4df5-93a8-a2659ea45160 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/groups/588218385e8546eab37eb1542ac0d3a9/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:54.851435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:54.853364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33a745-feeb-4df5-93a8-a2659ea45160 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fc1137956e4fdb9e9aeb9740c88d9c, group_id=588218385e8546eab37eb1542ac0d3a9, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:54.856489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33a745-feeb-4df5-93a8-a2659ea45160 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:54.876220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 200/405] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Jun 15 15:36:54 2022] PUT /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/groups/588218385e8546eab37eb1542ac0d3a9/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:54.877203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a745-feeb-4df5-93a8-a2659ea45160 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:54.947645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948b803-06f4-46b3-b105-66484b1011d5 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:54.985972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9be5e8-dc9b-4c8d-9249-1e829d83bf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:54.987108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9be5e8-dc9b-4c8d-9249-1e829d83bf44 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:54.987553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9be5e8-dc9b-4c8d-9249-1e829d83bf44 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:54.987972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9be5e8-dc9b-4c8d-9249-1e829d83bf44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.001961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9be5e8-dc9b-4c8d-9249-1e829d83bf44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:55.003300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a745-feeb-4df5-93a8-a2659ea45160 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-9c9be5e8-dc9b-4c8d-9249-1e829d83bf44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:55.012126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:55.013350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:55.014351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 206/406] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:55.021225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9be5e8-dc9b-4c8d-9249-1e829d83bf44 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:55.057792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270d7d-1929-4448-8811-d147a05a7f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:55.059426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270d7d-1929-4448-8811-d147a05a7f8f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:55.060161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270d7d-1929-4448-8811-d147a05a7f8f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:55.060622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270d7d-1929-4448-8811-d147a05a7f8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.077375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96270d7d-1929-4448-8811-d147a05a7f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-96270d7d-1929-4448-8811-d147a05a7f8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:55.089460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:55.090426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:55.091274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:55.092735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 201/407] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:55.098532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270d7d-1929-4448-8811-d147a05a7f8f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:55.133479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:55.152564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9be5e8-dc9b-4c8d-9249-1e829d83bf44 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:55.159968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:55.190691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad43682-7db3-4122-b96d-6ddf866beca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:55.191883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad43682-7db3-4122-b96d-6ddf866beca7 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:55.192376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad43682-7db3-4122-b96d-6ddf866beca7 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:55.192852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad43682-7db3-4122-b96d-6ddf866beca7 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.193832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad43682-7db3-4122-b96d-6ddf866beca7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:55.194341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad43682-7db3-4122-b96d-6ddf866beca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:55.196042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad43682-7db3-4122-b96d-6ddf866beca7 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:55.205921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 207/408] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:55.211117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad43682-7db3-4122-b96d-6ddf866beca7 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:55.219581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270d7d-1929-4448-8811-d147a05a7f8f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:55.258934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17363-fda4-4d5e-a9d5-c3926d296efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:55.259611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17363-fda4-4d5e-a9d5-c3926d296efe None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:55.259894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17363-fda4-4d5e-a9d5-c3926d296efe None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:55.260171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17363-fda4-4d5e-a9d5-c3926d296efe None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.261028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17363-fda4-4d5e-a9d5-c3926d296efe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:55.261328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17363-fda4-4d5e-a9d5-c3926d296efe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:55.262720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17363-fda4-4d5e-a9d5-c3926d296efe None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:55.274985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 202/409] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:55.279216 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17363-fda4-4d5e-a9d5-c3926d296efe None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:55.325294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad43682-7db3-4122-b96d-6ddf866beca7 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:55.362750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd78720-0ede-47b2-af75-4d49ca4ccbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:55.364003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd78720-0ede-47b2-af75-4d49ca4ccbb0 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:55.364555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd78720-0ede-47b2-af75-4d49ca4ccbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:55.365165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd78720-0ede-47b2-af75-4d49ca4ccbb0 None admin] PATH_INFO: `/v3/groups/dfb27ed5df9c4aa7915a854083d47857` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.366607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd78720-0ede-47b2-af75-4d49ca4ccbb0 None admin] RBAC: Authorizing `identity:get_group(group_id=dfb27ed5df9c4aa7915a854083d47857)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:55.369017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd78720-0ede-47b2-af75-4d49ca4ccbb0 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:55.371344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 208/410] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:55 2022] GET /identity/v3/groups/dfb27ed5df9c4aa7915a854083d47857 => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:55.376286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd78720-0ede-47b2-af75-4d49ca4ccbb0 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:55.387812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17363-fda4-4d5e-a9d5-c3926d296efe None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:55.394430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:55.403518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:55.426522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894085be-0e82-41c5-9bd8-cbcd774bf97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:55.427440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894085be-0e82-41c5-9bd8-cbcd774bf97f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:55.427815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894085be-0e82-41c5-9bd8-cbcd774bf97f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:55.428004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894085be-0e82-41c5-9bd8-cbcd774bf97f None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.429185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894085be-0e82-41c5-9bd8-cbcd774bf97f None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:55.431618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894085be-0e82-41c5-9bd8-cbcd774bf97f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:55.438140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 203/411] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:55 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:55.439063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894085be-0e82-41c5-9bd8-cbcd774bf97f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:55.490344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd78720-0ede-47b2-af75-4d49ca4ccbb0 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:55.561608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894085be-0e82-41c5-9bd8-cbcd774bf97f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:55.600384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95c0a3-9f81-4a18-a7d6-f4c83e69438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:55.601516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef4a79-49a0-4ce9-94c7-89308ef432a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:55.602946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef4a79-49a0-4ce9-94c7-89308ef432a2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:55.603232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef4a79-49a0-4ce9-94c7-89308ef432a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:55.603508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef4a79-49a0-4ce9-94c7-89308ef432a2 None admin] PATH_INFO: `/v3/projects/87109ec03f124710b1afe403b1e173c0` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.604414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c0a3-9f81-4a18-a7d6-f4c83e69438e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:55.604696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c0a3-9f81-4a18-a7d6-f4c83e69438e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:55.604967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c0a3-9f81-4a18-a7d6-f4c83e69438e None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.606207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef4a79-49a0-4ce9-94c7-89308ef432a2 None admin] RBAC: Authorizing `identity:get_project(project_id=87109ec03f124710b1afe403b1e173c0)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:55.607069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95c0a3-9f81-4a18-a7d6-f4c83e69438e None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:55.609897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef4a79-49a0-4ce9-94c7-89308ef432a2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:55.611812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 209/412] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:55 2022] GET /identity/v3/projects/87109ec03f124710b1afe403b1e173c0 => generated 344 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:55.611943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95c0a3-9f81-4a18-a7d6-f4c83e69438e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:55.613627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 204/413] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:55 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:55.619422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef4a79-49a0-4ce9-94c7-89308ef432a2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:55.632384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95c0a3-9f81-4a18-a7d6-f4c83e69438e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:55.723126 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:55.791230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95c0a3-9f81-4a18-a7d6-f4c83e69438e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:55.830235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffccb58-90a2-4c41-b7e9-04b130285357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:55.831451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffccb58-90a2-4c41-b7e9-04b130285357 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:55.831738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffccb58-90a2-4c41-b7e9-04b130285357 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:55.832017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffccb58-90a2-4c41-b7e9-04b130285357 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/groups/588218385e8546eab37eb1542ac0d3a9/roles/7fb980f85eb04293a0d1dba515bf0a2c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.834134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:55.835994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffccb58-90a2-4c41-b7e9-04b130285357 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fc1137956e4fdb9e9aeb9740c88d9c, group_id=588218385e8546eab37eb1542ac0d3a9, role_id=7fb980f85eb04293a0d1dba515bf0a2c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:55.838012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffccb58-90a2-4c41-b7e9-04b130285357 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:55.858809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 205/414] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Jun 15 15:36:55 2022] PUT /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c/groups/588218385e8546eab37eb1542ac0d3a9/roles/7fb980f85eb04293a0d1dba515bf0a2c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:55.887275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef4a79-49a0-4ce9-94c7-89308ef432a2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:55.951103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ef2c2-d81c-4658-a515-838fa0a32972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:55.952013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ef2c2-d81c-4658-a515-838fa0a32972 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:55.952296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ef2c2-d81c-4658-a515-838fa0a32972 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:55.952575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ef2c2-d81c-4658-a515-838fa0a32972 None admin] PATH_INFO: `/v3/projects/87109ec03f124710b1afe403b1e173c0/groups/dfb27ed5df9c4aa7915a854083d47857/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:55.960451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ef2c2-d81c-4658-a515-838fa0a32972 None admin] RBAC: Authorizing `identity:create_grant(project_id=87109ec03f124710b1afe403b1e173c0, group_id=dfb27ed5df9c4aa7915a854083d47857, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:55.962483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ef2c2-d81c-4658-a515-838fa0a32972 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:55.985771 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 210/415] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Jun 15 15:36:55 2022] PUT /identity/v3/projects/87109ec03f124710b1afe403b1e173c0/groups/dfb27ed5df9c4aa7915a854083d47857/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:36:56.171787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083fb06-a5ab-4622-84f6-538f3c2fa753 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:56.172863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083fb06-a5ab-4622-84f6-538f3c2fa753 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:56.173361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083fb06-a5ab-4622-84f6-538f3c2fa753 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:56.175442 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 206/416] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:56.181528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa02677-d733-4ac6-b049-3a2c097d1ff6 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:56.182196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa02677-d733-4ac6-b049-3a2c097d1ff6 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:56.182686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa02677-d733-4ac6-b049-3a2c097d1ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:56.190021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a4853-fc22-4078-bbb7-10b9a24a68be None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:56.190332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a4853-fc22-4078-bbb7-10b9a24a68be None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:56.190592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a4853-fc22-4078-bbb7-10b9a24a68be None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:56.192269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 207/417] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:56.198958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61277b2b-b9b0-4ca7-b42e-e614d11a92e3 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:56.199253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61277b2b-b9b0-4ca7-b42e-e614d11a92e3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:56.199535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61277b2b-b9b0-4ca7-b42e-e614d11a92e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:56.203476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4aa02677-d733-4ac6-b049-3a2c097d1ff6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:56.205009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4aa02677-d733-4ac6-b049-3a2c097d1ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:56.228337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-61277b2b-b9b0-4ca7-b42e-e614d11a92e3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:56.230153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-61277b2b-b9b0-4ca7-b42e-e614d11a92e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:56.295835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 211/418] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:56.324144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa02677-d733-4ac6-b049-3a2c097d1ff6 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:56.325699 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 208/419] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:36:56.348429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61277b2b-b9b0-4ca7-b42e-e614d11a92e3 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:56.393616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:56.471127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:56.516612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61277b2b-b9b0-4ca7-b42e-e614d11a92e3 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:56.520940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa02677-d733-4ac6-b049-3a2c097d1ff6 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:56.567225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06228fc-7d5c-4272-b1d9-f4500f4f9f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:56.568131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06228fc-7d5c-4272-b1d9-f4500f4f9f07 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:56.568447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06228fc-7d5c-4272-b1d9-f4500f4f9f07 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:56.569129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06228fc-7d5c-4272-b1d9-f4500f4f9f07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:56.580814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06228fc-7d5c-4272-b1d9-f4500f4f9f07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-a06228fc-7d5c-4272-b1d9-f4500f4f9f07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:56.590480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:56.591411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:36:56.592202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 209/420] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:56.597697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06228fc-7d5c-4272-b1d9-f4500f4f9f07 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:56.600956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a416520-8da3-4167-b3be-cfd12a9bf6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:56.602001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a416520-8da3-4167-b3be-cfd12a9bf6f0 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:56.603060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a416520-8da3-4167-b3be-cfd12a9bf6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:56.603519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a416520-8da3-4167-b3be-cfd12a9bf6f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:56.627611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a416520-8da3-4167-b3be-cfd12a9bf6f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-4a416520-8da3-4167-b3be-cfd12a9bf6f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:56.637906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:56.638829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:56.639738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 212/421] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:56.646399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a416520-8da3-4167-b3be-cfd12a9bf6f0 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:56.761639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a416520-8da3-4167-b3be-cfd12a9bf6f0 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:56.795220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:56.801397 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a34fdb-9a91-4482-8bb9-297bb66d0fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:56.802069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a34fdb-9a91-4482-8bb9-297bb66d0fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:56.802352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a34fdb-9a91-4482-8bb9-297bb66d0fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:56.802626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a34fdb-9a91-4482-8bb9-297bb66d0fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:56.803434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a34fdb-9a91-4482-8bb9-297bb66d0fb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:56.803741 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a34fdb-9a91-4482-8bb9-297bb66d0fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:56.805122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a34fdb-9a91-4482-8bb9-297bb66d0fb9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:56.816112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 213/422] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:56.822695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a34fdb-9a91-4482-8bb9-297bb66d0fb9 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:56.840182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06228fc-7d5c-4272-b1d9-f4500f4f9f07 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:56.871087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:56.880090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de86b0-e124-48ae-aba9-f1bb576a0d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:56.880776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de86b0-e124-48ae-aba9-f1bb576a0d3c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:56.881066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de86b0-e124-48ae-aba9-f1bb576a0d3c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:56.881355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de86b0-e124-48ae-aba9-f1bb576a0d3c None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:56.882111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de86b0-e124-48ae-aba9-f1bb576a0d3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:56.882487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de86b0-e124-48ae-aba9-f1bb576a0d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:56.884520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de86b0-e124-48ae-aba9-f1bb576a0d3c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:56.894409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 210/423] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:56.900204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de86b0-e124-48ae-aba9-f1bb576a0d3c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:57.032727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a34fdb-9a91-4482-8bb9-297bb66d0fb9 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:57.071349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8429f5-8cfc-45c3-95a5-8a0d2bda2cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:57.072326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8429f5-8cfc-45c3-95a5-8a0d2bda2cc5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:57.072749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8429f5-8cfc-45c3-95a5-8a0d2bda2cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:57.073040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8429f5-8cfc-45c3-95a5-8a0d2bda2cc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:57.084511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de86b0-e124-48ae-aba9-f1bb576a0d3c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:57.144566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:57.148871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8429f5-8cfc-45c3-95a5-8a0d2bda2cc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:57.151077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8429f5-8cfc-45c3-95a5-8a0d2bda2cc5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:57.179441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab6e32-caf0-4961-b9ea-b70d3b4f3c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:57.181375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab6e32-caf0-4961-b9ea-b70d3b4f3c8b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:57.181865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab6e32-caf0-4961-b9ea-b70d3b4f3c8b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:57.182322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab6e32-caf0-4961-b9ea-b70d3b4f3c8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:57.263535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 214/424] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:57.268982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8429f5-8cfc-45c3-95a5-8a0d2bda2cc5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:57.274709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:57.357848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab6e32-caf0-4961-b9ea-b70d3b4f3c8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:57.359656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab6e32-caf0-4961-b9ea-b70d3b4f3c8b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:57.372921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:57.381239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8429f5-8cfc-45c3-95a5-8a0d2bda2cc5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:57.427050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691a057-6cd4-4198-8f74-a86d1ffd2654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:57.427050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691a057-6cd4-4198-8f74-a86d1ffd2654 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:57.427050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691a057-6cd4-4198-8f74-a86d1ffd2654 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:57.427050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691a057-6cd4-4198-8f74-a86d1ffd2654 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:57.427447 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691a057-6cd4-4198-8f74-a86d1ffd2654 None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:57.427447 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691a057-6cd4-4198-8f74-a86d1ffd2654 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:57.428783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 215/425] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:57 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:57.440170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691a057-6cd4-4198-8f74-a86d1ffd2654 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:57.483354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 211/426] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 15:36:57.488453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab6e32-caf0-4961-b9ea-b70d3b4f3c8b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:57.555121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:57.597592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab6e32-caf0-4961-b9ea-b70d3b4f3c8b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:57.640704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf072d-c42f-4531-8c3c-9b327100cdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:57.641580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf072d-c42f-4531-8c3c-9b327100cdf6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:57.641865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf072d-c42f-4531-8c3c-9b327100cdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:57.642144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf072d-c42f-4531-8c3c-9b327100cdf6 None admin] PATH_INFO: `/v3/projects/54d37e7eb5474cf387beec6e56303b4f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:57.643555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf072d-c42f-4531-8c3c-9b327100cdf6 None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e7eb5474cf387beec6e56303b4f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:57.645003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf072d-c42f-4531-8c3c-9b327100cdf6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:57.650137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 212/427] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:57 2022] GET /identity/v3/projects/54d37e7eb5474cf387beec6e56303b4f => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:57.654300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf072d-c42f-4531-8c3c-9b327100cdf6 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:57.722009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691a057-6cd4-4198-8f74-a86d1ffd2654 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:57.786606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0e931-baf5-431d-acd0-1848ef86b449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:57.787490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0e931-baf5-431d-acd0-1848ef86b449 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:57.787772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0e931-baf5-431d-acd0-1848ef86b449 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:57.788049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0e931-baf5-431d-acd0-1848ef86b449 None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:57.795761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0e931-baf5-431d-acd0-1848ef86b449 None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:57.798089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0e931-baf5-431d-acd0-1848ef86b449 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:57.800964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 216/428] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:57 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:57.803187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c7d31-8dbe-40a0-be04-149c6294184f None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:57.803751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c7d31-8dbe-40a0-be04-149c6294184f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:57.804233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c7d31-8dbe-40a0-be04-149c6294184f None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:57.806275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 217/429] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:57.807253 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c7d31-8dbe-40a0-be04-149c6294184f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:57.828474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf072d-c42f-4531-8c3c-9b327100cdf6 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:57.847729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:57.858558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:57.866950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2e84b-04e4-4c0e-bae7-4689fed07776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:57.867706 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2e84b-04e4-4c0e-bae7-4689fed07776 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:57.867996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2e84b-04e4-4c0e-bae7-4689fed07776 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:57.868272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2e84b-04e4-4c0e-bae7-4689fed07776 None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:57.869457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2e84b-04e4-4c0e-bae7-4689fed07776 None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:57.871042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2e84b-04e4-4c0e-bae7-4689fed07776 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:57.877636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 213/430] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:57 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:57.879028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e4e1b-f104-49a0-85e6-d0eec6410032 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:57.879306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e4e1b-f104-49a0-85e6-d0eec6410032 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:57.879593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e4e1b-f104-49a0-85e6-d0eec6410032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:57.897592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-902e4e1b-f104-49a0-85e6-d0eec6410032 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:57.898848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-902e4e1b-f104-49a0-85e6-d0eec6410032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:57.935222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c7d31-8dbe-40a0-be04-149c6294184f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:57.974642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38b9d4-4269-42da-ba62-af286e987527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:57.975690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38b9d4-4269-42da-ba62-af286e987527 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:57.976177 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38b9d4-4269-42da-ba62-af286e987527 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:57.976735 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38b9d4-4269-42da-ba62-af286e987527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:57.978324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38b9d4-4269-42da-ba62-af286e987527 None admin] RBAC: Adding query filter params (group.id=588218385e8546eab37eb1542ac0d3a9, role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=54d37e7eb5474cf387beec6e56303b4f) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:57.978975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38b9d4-4269-42da-ba62-af286e987527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:57.980818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38b9d4-4269-42da-ba62-af286e987527 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:57.982989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 214/431] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:57.983868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e4e1b-f104-49a0-85e6-d0eec6410032 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:57.988561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 218/432] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:36:57 2022] GET /identity/v3/role_assignments?group.id=588218385e8546eab37eb1542ac0d3a9&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:58.033857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38b9d4-4269-42da-ba62-af286e987527 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:58.110440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e4e1b-f104-49a0-85e6-d0eec6410032 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:58.152839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:58.162057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f7e77-4b28-420d-9217-5657c318916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:58.162885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f7e77-4b28-420d-9217-5657c318916d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.163171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f7e77-4b28-420d-9217-5657c318916d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.163450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f7e77-4b28-420d-9217-5657c318916d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.175221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f7e77-4b28-420d-9217-5657c318916d None admin] RBAC: Adding query filter params (group.id=588218385e8546eab37eb1542ac0d3a9, role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=54d37e7eb5474cf387beec6e56303b4f) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:58.175602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f7e77-4b28-420d-9217-5657c318916d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:58.177100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f7e77-4b28-420d-9217-5657c318916d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:58.184921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 215/433] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:36:57 2022] GET /identity/v3/role_assignments?group.id=588218385e8546eab37eb1542ac0d3a9&scope.project.id=54d37e7eb5474cf387beec6e56303b4f&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:58.198867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:58.207281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38b9d4-4269-42da-ba62-af286e987527 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:58.246816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a825f-b570-4eaa-94ec-0c9c93f19a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:58.247730 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a825f-b570-4eaa-94ec-0c9c93f19a47 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.248014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a825f-b570-4eaa-94ec-0c9c93f19a47 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.248293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a825f-b570-4eaa-94ec-0c9c93f19a47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.261416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a825f-b570-4eaa-94ec-0c9c93f19a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-699a825f-b570-4eaa-94ec-0c9c93f19a47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:58.269685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:58.270649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:58.271757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 219/434] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:58.280448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f7e77-4b28-420d-9217-5657c318916d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:58.387226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab176fdf-5568-4372-900c-1a3f491524c2 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.387534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab176fdf-5568-4372-900c-1a3f491524c2 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.387876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab176fdf-5568-4372-900c-1a3f491524c2 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.393531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 220/435] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:36:58.395749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb6ff1-6ca2-45f7-afeb-98883fe9ec55 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.396062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb6ff1-6ca2-45f7-afeb-98883fe9ec55 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.396319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb6ff1-6ca2-45f7-afeb-98883fe9ec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.417772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f7e77-4b28-420d-9217-5657c318916d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:58.420368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-feeb6ff1-6ca2-45f7-afeb-98883fe9ec55 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:58.422224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-feeb6ff1-6ca2-45f7-afeb-98883fe9ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:58.471396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e750-a0ce-4c74-af6e-01e0c22ebdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:58.472562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e750-a0ce-4c74-af6e-01e0c22ebdeb None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.473048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e750-a0ce-4c74-af6e-01e0c22ebdeb None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.473525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e750-a0ce-4c74-af6e-01e0c22ebdeb None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.474436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635e750-a0ce-4c74-af6e-01e0c22ebdeb None admin] RBAC: Adding query filter params (name=member) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:58.474769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635e750-a0ce-4c74-af6e-01e0c22ebdeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:58.476148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635e750-a0ce-4c74-af6e-01e0c22ebdeb None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:58.486954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 216/436] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:36:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:58.493558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e750-a0ce-4c74-af6e-01e0c22ebdeb None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:58.518785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 221/437] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:58.544227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb6ff1-6ca2-45f7-afeb-98883fe9ec55 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:58.560214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:58.626404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e750-a0ce-4c74-af6e-01e0c22ebdeb None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:58.655163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:58.665569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf84f2f-5c28-4429-aba5-a320430c95a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:58.666562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf84f2f-5c28-4429-aba5-a320430c95a6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.667089 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf84f2f-5c28-4429-aba5-a320430c95a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.667590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf84f2f-5c28-4429-aba5-a320430c95a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.671340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb6ff1-6ca2-45f7-afeb-98883fe9ec55 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:58.716327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f1623-b52d-4a8a-aa52-c1a29f2b9102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:58.717146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f1623-b52d-4a8a-aa52-c1a29f2b9102 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.717431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f1623-b52d-4a8a-aa52-c1a29f2b9102 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.717760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f1623-b52d-4a8a-aa52-c1a29f2b9102 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.729763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f1623-b52d-4a8a-aa52-c1a29f2b9102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-c66f1623-b52d-4a8a-aa52-c1a29f2b9102 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:58.737933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:58.739073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:36:58.740056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:58.740695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 222/438] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:36:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:36:58.741846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e482216-d1b6-4587-b9c1-348f3d537bbd None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.742134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e482216-d1b6-4587-b9c1-348f3d537bbd None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.742413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e482216-d1b6-4587-b9c1-348f3d537bbd None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.744967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 223/439] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:36:58.746389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e482216-d1b6-4587-b9c1-348f3d537bbd None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:58.757928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf84f2f-5c28-4429-aba5-a320430c95a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:58.759612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf84f2f-5c28-4429-aba5-a320430c95a6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:58.823022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:58.860498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e482216-d1b6-4587-b9c1-348f3d537bbd None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:58.876298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 217/440] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:58.877682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a49c7-a08e-490f-9686-f0cdb1181e04 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.877980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a49c7-a08e-490f-9686-f0cdb1181e04 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.878259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a49c7-a08e-490f-9686-f0cdb1181e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.893979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d53a49c7-a08e-490f-9686-f0cdb1181e04 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:36:58.895187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d53a49c7-a08e-490f-9686-f0cdb1181e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:36:58.897605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44393c-b8e5-484c-aac4-5802ce5e8f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:58.898269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44393c-b8e5-484c-aac4-5802ce5e8f61 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:58.898556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44393c-b8e5-484c-aac4-5802ce5e8f61 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:58.899419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44393c-b8e5-484c-aac4-5802ce5e8f61 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:58.900681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44393c-b8e5-484c-aac4-5802ce5e8f61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:58.901065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44393c-b8e5-484c-aac4-5802ce5e8f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:58.902461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44393c-b8e5-484c-aac4-5802ce5e8f61 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:58.910330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 224/441] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:36:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:36:58.910330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44393c-b8e5-484c-aac4-5802ce5e8f61 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:58.921602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:58.965661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 218/442] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:36:58.966491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a49c7-a08e-490f-9686-f0cdb1181e04 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.023100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44393c-b8e5-484c-aac4-5802ce5e8f61 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.047974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:59.057582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f1390-3600-4e75-a580-2ac09168e25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.058438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1390-3600-4e75-a580-2ac09168e25d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.058723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1390-3600-4e75-a580-2ac09168e25d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.059025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1390-3600-4e75-a580-2ac09168e25d None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.060226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f1390-3600-4e75-a580-2ac09168e25d None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:59.061676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f1390-3600-4e75-a580-2ac09168e25d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:59.063389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 225/443] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:58 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:59.064145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f1390-3600-4e75-a580-2ac09168e25d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.070643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a49c7-a08e-490f-9686-f0cdb1181e04 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.104467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b123fc7-312e-449b-b2b7-29af782d9b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.105109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b123fc7-312e-449b-b2b7-29af782d9b66 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.105396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b123fc7-312e-449b-b2b7-29af782d9b66 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.105681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b123fc7-312e-449b-b2b7-29af782d9b66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.137825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:59.163765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f1390-3600-4e75-a580-2ac09168e25d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.173532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b123fc7-312e-449b-b2b7-29af782d9b66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:59.174956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b123fc7-312e-449b-b2b7-29af782d9b66 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:59.198497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c856-0499-486b-bea1-e5c834e18245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.199309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c856-0499-486b-bea1-e5c834e18245 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.199598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c856-0499-486b-bea1-e5c834e18245 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.199881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c856-0499-486b-bea1-e5c834e18245 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.209803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33c856-0499-486b-bea1-e5c834e18245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-5b33c856-0499-486b-bea1-e5c834e18245 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:36:59.217458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:36:59.218326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 226/444] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:36:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:36:59.219499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c856-0499-486b-bea1-e5c834e18245 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.259279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:59.277642 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 219/445] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:59.278448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b123fc7-312e-449b-b2b7-29af782d9b66 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.318806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c856-0499-486b-bea1-e5c834e18245 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.351665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42babfb1-d98a-497a-9f32-188f6eebf6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.353037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42babfb1-d98a-497a-9f32-188f6eebf6ab None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.353634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42babfb1-d98a-497a-9f32-188f6eebf6ab None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.353634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42babfb1-d98a-497a-9f32-188f6eebf6ab None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.354787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42babfb1-d98a-497a-9f32-188f6eebf6ab None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:59.356251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42babfb1-d98a-497a-9f32-188f6eebf6ab None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:59.357867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 227/446] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:59 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:59.358620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42babfb1-d98a-497a-9f32-188f6eebf6ab None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.361178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:59.379427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b123fc7-312e-449b-b2b7-29af782d9b66 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.412671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44393d-766f-4c4e-93a5-9d4a3242ec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.413337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44393d-766f-4c4e-93a5-9d4a3242ec1b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.413625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44393d-766f-4c4e-93a5-9d4a3242ec1b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.413908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44393d-766f-4c4e-93a5-9d4a3242ec1b None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.415210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44393d-766f-4c4e-93a5-9d4a3242ec1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:59.415524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44393d-766f-4c4e-93a5-9d4a3242ec1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:59.416884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44393d-766f-4c4e-93a5-9d4a3242ec1b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:59.424937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 220/447] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:36:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:59.425655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44393d-766f-4c4e-93a5-9d4a3242ec1b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.459493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42babfb1-d98a-497a-9f32-188f6eebf6ab None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.473796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:59.493356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28e845-5add-4742-8ce1-bc9211462515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.494202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28e845-5add-4742-8ce1-bc9211462515 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.494490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28e845-5add-4742-8ce1-bc9211462515 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.494797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28e845-5add-4742-8ce1-bc9211462515 None admin] PATH_INFO: `/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.496006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28e845-5add-4742-8ce1-bc9211462515 None admin] RBAC: Authorizing `identity:get_project(project_id=25fc1137956e4fdb9e9aeb9740c88d9c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:59.497457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28e845-5add-4742-8ce1-bc9211462515 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:59.499171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 228/448] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:59 2022] GET /identity/v3/projects/25fc1137956e4fdb9e9aeb9740c88d9c => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:59.500367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28e845-5add-4742-8ce1-bc9211462515 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.530851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44393d-766f-4c4e-93a5-9d4a3242ec1b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.563833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e984db0-7cf3-4793-89f9-31cd89fb403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.564480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e984db0-7cf3-4793-89f9-31cd89fb403c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.564765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e984db0-7cf3-4793-89f9-31cd89fb403c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.565043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e984db0-7cf3-4793-89f9-31cd89fb403c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.565870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e984db0-7cf3-4793-89f9-31cd89fb403c None admin] RBAC: Adding query filter params (group.id=588218385e8546eab37eb1542ac0d3a9, role.id=4d4438a26f504e9f84ba62dc395f926e, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:59.566181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e984db0-7cf3-4793-89f9-31cd89fb403c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:59.566704 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:59.567627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e984db0-7cf3-4793-89f9-31cd89fb403c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:59.573434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 221/449] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:36:59 2022] GET /identity/v3/role_assignments?group.id=588218385e8546eab37eb1542ac0d3a9&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=4d4438a26f504e9f84ba62dc395f926e => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:59.574211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e984db0-7cf3-4793-89f9-31cd89fb403c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.612031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28e845-5add-4742-8ce1-bc9211462515 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.652635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d144e-86cf-47a4-a4dd-e7fe5a5b69f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.653296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d144e-86cf-47a4-a4dd-e7fe5a5b69f2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.653603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d144e-86cf-47a4-a4dd-e7fe5a5b69f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.653859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d144e-86cf-47a4-a4dd-e7fe5a5b69f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.697429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e984db0-7cf3-4793-89f9-31cd89fb403c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.708029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:59.730099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d144e-86cf-47a4-a4dd-e7fe5a5b69f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:59.731604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d144e-86cf-47a4-a4dd-e7fe5a5b69f2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:59.739453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869443ee-c67d-4b0e-9750-9683971afe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.740319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869443ee-c67d-4b0e-9750-9683971afe98 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.740604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869443ee-c67d-4b0e-9750-9683971afe98 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.740888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869443ee-c67d-4b0e-9750-9683971afe98 None admin] PATH_INFO: `/v3/groups/588218385e8546eab37eb1542ac0d3a9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.742114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869443ee-c67d-4b0e-9750-9683971afe98 None admin] RBAC: Authorizing `identity:get_group(group_id=588218385e8546eab37eb1542ac0d3a9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:59.744147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869443ee-c67d-4b0e-9750-9683971afe98 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:59.745779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 222/450] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:36:59 2022] GET /identity/v3/groups/588218385e8546eab37eb1542ac0d3a9 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:59.751959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869443ee-c67d-4b0e-9750-9683971afe98 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.808209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:36:59.852390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 229/451] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:36:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:36:59.859124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d144e-86cf-47a4-a4dd-e7fe5a5b69f2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:36:59.875905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869443ee-c67d-4b0e-9750-9683971afe98 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:36:59.917658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5ed6f-9aea-4f16-89ff-adb0fe3f0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:36:59.918314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5ed6f-9aea-4f16-89ff-adb0fe3f0d82 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:36:59.918598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5ed6f-9aea-4f16-89ff-adb0fe3f0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:36:59.918912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5ed6f-9aea-4f16-89ff-adb0fe3f0d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:36:59.919779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5ed6f-9aea-4f16-89ff-adb0fe3f0d82 None admin] RBAC: Adding query filter params (group.id=588218385e8546eab37eb1542ac0d3a9, role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:36:59.920068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5ed6f-9aea-4f16-89ff-adb0fe3f0d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:36:59.921513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5ed6f-9aea-4f16-89ff-adb0fe3f0d82 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:36:59.928617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 223/452] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:36:59 2022] GET /identity/v3/role_assignments?group.id=588218385e8546eab37eb1542ac0d3a9&scope.project.id=25fc1137956e4fdb9e9aeb9740c88d9c&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:36:59.974668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d144e-86cf-47a4-a4dd-e7fe5a5b69f2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:00.011434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f9fad-b0a0-4036-88ec-4eb6ef16af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:00.012306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f9fad-b0a0-4036-88ec-4eb6ef16af6d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:00.012587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f9fad-b0a0-4036-88ec-4eb6ef16af6d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:00.012861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f9fad-b0a0-4036-88ec-4eb6ef16af6d None admin] PATH_INFO: `/v3/projects/87109ec03f124710b1afe403b1e173c0` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:00.014169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f9fad-b0a0-4036-88ec-4eb6ef16af6d None admin] RBAC: Authorizing `identity:get_project(project_id=87109ec03f124710b1afe403b1e173c0)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:00.015682 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f9fad-b0a0-4036-88ec-4eb6ef16af6d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:00.017423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 230/453] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:36:59 2022] GET /identity/v3/projects/87109ec03f124710b1afe403b1e173c0 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:00.023324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5ed6f-9aea-4f16-89ff-adb0fe3f0d82 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:00.067597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:00.128054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5ed6f-9aea-4f16-89ff-adb0fe3f0d82 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:00.170221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d5a4f-7863-4629-b14d-4dac01347c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:00.171118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d5a4f-7863-4629-b14d-4dac01347c6a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:00.171398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d5a4f-7863-4629-b14d-4dac01347c6a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:00.171677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d5a4f-7863-4629-b14d-4dac01347c6a None admin] PATH_INFO: `/v3/groups/dfb27ed5df9c4aa7915a854083d47857` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:00.172850 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d5a4f-7863-4629-b14d-4dac01347c6a None admin] RBAC: Authorizing `identity:get_group(group_id=dfb27ed5df9c4aa7915a854083d47857)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:00.174276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d5a4f-7863-4629-b14d-4dac01347c6a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:00.175917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 224/454] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Jun 15 15:37:00 2022] GET /identity/v3/groups/dfb27ed5df9c4aa7915a854083d47857 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:00.181831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f9fad-b0a0-4036-88ec-4eb6ef16af6d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:00.225831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:00.287414 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f9fad-b0a0-4036-88ec-4eb6ef16af6d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:00.320745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de78460-2ffa-4f05-a5c9-bd67234ae947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:00.321390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de78460-2ffa-4f05-a5c9-bd67234ae947 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:00.321671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de78460-2ffa-4f05-a5c9-bd67234ae947 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:00.321947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de78460-2ffa-4f05-a5c9-bd67234ae947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:00.322786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de78460-2ffa-4f05-a5c9-bd67234ae947 None admin] RBAC: Adding query filter params (group.id=dfb27ed5df9c4aa7915a854083d47857, role.id=433478cc318a467c931f13a757f161fa, scope.project.id=87109ec03f124710b1afe403b1e173c0) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:00.323096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de78460-2ffa-4f05-a5c9-bd67234ae947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:00.324534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de78460-2ffa-4f05-a5c9-bd67234ae947 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:00.330162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 231/455] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Jun 15 15:37:00 2022] GET /identity/v3/role_assignments?group.id=dfb27ed5df9c4aa7915a854083d47857&scope.project.id=87109ec03f124710b1afe403b1e173c0&role.id=433478cc318a467c931f13a757f161fa => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:03.219152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a0540-93cf-43ee-a026-a85a05215393 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.219664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a0540-93cf-43ee-a026-a85a05215393 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.219741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a0540-93cf-43ee-a026-a85a05215393 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.221341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 225/456] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:03.226604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36777150-6c92-431e-8448-c0e25ff6c442 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.227611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36777150-6c92-431e-8448-c0e25ff6c442 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.227711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36777150-6c92-431e-8448-c0e25ff6c442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.239915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c573e-6ad1-48a6-8e1e-eade93347f29 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.240225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c573e-6ad1-48a6-8e1e-eade93347f29 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.240486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c573e-6ad1-48a6-8e1e-eade93347f29 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.242214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 226/457] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:03.250215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb416afb-6377-4980-a8c5-d8a6297a6445 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.250518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb416afb-6377-4980-a8c5-d8a6297a6445 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.250835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb416afb-6377-4980-a8c5-d8a6297a6445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.267833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-36777150-6c92-431e-8448-c0e25ff6c442 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:03.269042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-36777150-6c92-431e-8448-c0e25ff6c442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:03.290911 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-bb416afb-6377-4980-a8c5-d8a6297a6445 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:03.292445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-bb416afb-6377-4980-a8c5-d8a6297a6445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:03.413112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 227/458] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:03.420339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 232/459] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:03.420455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd0f7-a703-4b6d-bacf-91167b85dfe6 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.420455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd0f7-a703-4b6d-bacf-91167b85dfe6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.420455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd0f7-a703-4b6d-bacf-91167b85dfe6 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.425365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 228/460] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:03.427169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ffde2-167e-44a8-b42e-352d06dbf3c1 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.427454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ffde2-167e-44a8-b42e-352d06dbf3c1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.427733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ffde2-167e-44a8-b42e-352d06dbf3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.444169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6f0ffde2-167e-44a8-b42e-352d06dbf3c1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:03.447470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6f0ffde2-167e-44a8-b42e-352d06dbf3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:03.451288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd0f7-a703-4b6d-bacf-91167b85dfe6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:03.474236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:03.504012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:03.520649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 233/461] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:03.522113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ffde2-167e-44a8-b42e-352d06dbf3c1 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:03.590586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd0f7-a703-4b6d-bacf-91167b85dfe6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:03.630752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e450edb-7ac6-46fd-85fd-f89073944e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:03.631996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e450edb-7ac6-46fd-85fd-f89073944e48 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.632479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e450edb-7ac6-46fd-85fd-f89073944e48 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.633016 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e450edb-7ac6-46fd-85fd-f89073944e48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.644437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e450edb-7ac6-46fd-85fd-f89073944e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:03.646395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e450edb-7ac6-46fd-85fd-f89073944e48 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:03.660758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ffde2-167e-44a8-b42e-352d06dbf3c1 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-8e450edb-7ac6-46fd-85fd-f89073944e48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:03.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:03.666635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:03.667879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:03.667879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:03.667879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:03.667879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:03.667879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 229/462] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:03.668345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e450edb-7ac6-46fd-85fd-f89073944e48 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:03.700908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e3d02-a5a1-40e9-8868-25e6df2d0bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:03.701708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e3d02-a5a1-40e9-8868-25e6df2d0bc5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.701992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e3d02-a5a1-40e9-8868-25e6df2d0bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.702271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e3d02-a5a1-40e9-8868-25e6df2d0bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.713085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e3d02-a5a1-40e9-8868-25e6df2d0bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:03.714680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e3d02-a5a1-40e9-8868-25e6df2d0bc5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-456e3d02-a5a1-40e9-8868-25e6df2d0bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:03.733940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:03.734990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:03.736053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:03.736053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:03.736053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:03.736053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:03.736053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 234/463] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:03.736385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a4d79-92d7-4907-bbe9-ac61c8cf2ada None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.736596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a4d79-92d7-4907-bbe9-ac61c8cf2ada None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.736878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a4d79-92d7-4907-bbe9-ac61c8cf2ada None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.738286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 235/464] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:03.742204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a4d79-92d7-4907-bbe9-ac61c8cf2ada None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:03.759905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:03.771995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:03.783462 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e450edb-7ac6-46fd-85fd-f89073944e48 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:03.820965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5c0e5-d6c0-4c28-b8f6-30747660a386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:03.821762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5c0e5-d6c0-4c28-b8f6-30747660a386 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.822045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5c0e5-d6c0-4c28-b8f6-30747660a386 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.822318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5c0e5-d6c0-4c28-b8f6-30747660a386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.833201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5c0e5-d6c0-4c28-b8f6-30747660a386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:03.834650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5c0e5-d6c0-4c28-b8f6-30747660a386 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-30b5c0e5-d6c0-4c28-b8f6-30747660a386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:03.853897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:03.854770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:03.855635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:03.855635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:03.855635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:03.855635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:03.855635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 230/465] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:03.856039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5c0e5-d6c0-4c28-b8f6-30747660a386 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:03.864336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a4d79-92d7-4907-bbe9-ac61c8cf2ada None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:03.901239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f758b-4efb-44c3-b8c5-8b35a2e71c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:03.901902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f758b-4efb-44c3-b8c5-8b35a2e71c01 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.902187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f758b-4efb-44c3-b8c5-8b35a2e71c01 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.902465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f758b-4efb-44c3-b8c5-8b35a2e71c01 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.903231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f758b-4efb-44c3-b8c5-8b35a2e71c01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:03.903543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f758b-4efb-44c3-b8c5-8b35a2e71c01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:03.904927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f758b-4efb-44c3-b8c5-8b35a2e71c01 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:03.916693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 236/466] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:03.917899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724f53a-9032-4173-983e-b15c6c8c55dc None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:03.918189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724f53a-9032-4173-983e-b15c6c8c55dc None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:03.918469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724f53a-9032-4173-983e-b15c6c8c55dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:03.935096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2724f53a-9032-4173-983e-b15c6c8c55dc None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:03.936296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2724f53a-9032-4173-983e-b15c6c8c55dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:03.972571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5c0e5-d6c0-4c28-b8f6-30747660a386 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:04.009648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 237/467] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:04.009648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724f53a-9032-4173-983e-b15c6c8c55dc None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:04.011065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:04.026264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f41e4-b12f-4432-8c07-b73315e8067b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:04.027489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f41e4-b12f-4432-8c07-b73315e8067b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.028394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f41e4-b12f-4432-8c07-b73315e8067b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.028824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f41e4-b12f-4432-8c07-b73315e8067b None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.029967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f41e4-b12f-4432-8c07-b73315e8067b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:04.030677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f41e4-b12f-4432-8c07-b73315e8067b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:04.032624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f41e4-b12f-4432-8c07-b73315e8067b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:04.045363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 231/468] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:04.046517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f41e4-b12f-4432-8c07-b73315e8067b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:04.049356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:04.123716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724f53a-9032-4173-983e-b15c6c8c55dc None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:04.153510 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f41e4-b12f-4432-8c07-b73315e8067b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:04.160626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3de283-5b73-4f29-8c6b-f841dbfe1581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:04.161293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3de283-5b73-4f29-8c6b-f841dbfe1581 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.161696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3de283-5b73-4f29-8c6b-f841dbfe1581 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.161978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3de283-5b73-4f29-8c6b-f841dbfe1581 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.162710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3de283-5b73-4f29-8c6b-f841dbfe1581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:04.163050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3de283-5b73-4f29-8c6b-f841dbfe1581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:04.164596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3de283-5b73-4f29-8c6b-f841dbfe1581 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:04.176444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 238/469] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:04.177237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3de283-5b73-4f29-8c6b-f841dbfe1581 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:04.196710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b673f7-49a9-4085-88f6-333d5ce4e6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:04.197381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b673f7-49a9-4085-88f6-333d5ce4e6e1 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.197750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b673f7-49a9-4085-88f6-333d5ce4e6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.198027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b673f7-49a9-4085-88f6-333d5ce4e6e1 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.199302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b673f7-49a9-4085-88f6-333d5ce4e6e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:04.200876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b673f7-49a9-4085-88f6-333d5ce4e6e1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:04.223237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 232/470] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:37:04 2022] POST /identity/v3/users => generated 248 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:04.224141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b673f7-49a9-4085-88f6-333d5ce4e6e1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:04.271510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:04.306221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:04.326012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3de283-5b73-4f29-8c6b-f841dbfe1581 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:04.358814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b673f7-49a9-4085-88f6-333d5ce4e6e1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:04.365048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cb7a0-7f0f-48fb-b7a1-459988c1fc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:04.365854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb7a0-7f0f-48fb-b7a1-459988c1fc07 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.366148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb7a0-7f0f-48fb-b7a1-459988c1fc07 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.366968 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb7a0-7f0f-48fb-b7a1-459988c1fc07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.380874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cb7a0-7f0f-48fb-b7a1-459988c1fc07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:04.382363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cb7a0-7f0f-48fb-b7a1-459988c1fc07 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:04.394948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76866910-78e0-4bee-9961-d32149328cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:04.395634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76866910-78e0-4bee-9961-d32149328cca None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.395918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76866910-78e0-4bee-9961-d32149328cca None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.396483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76866910-78e0-4bee-9961-d32149328cca None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.397203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76866910-78e0-4bee-9961-d32149328cca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:04.399223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76866910-78e0-4bee-9961-d32149328cca None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-039cb7a0-7f0f-48fb-b7a1-459988c1fc07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:04.403154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:04.404012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:04.404859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:04.404859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:04.404859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:04.404859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:04.404859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 239/471] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:04.405529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cb7a0-7f0f-48fb-b7a1-459988c1fc07 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:04.421930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 233/472] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:37:04 2022] POST /identity/v3/users => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:04.425004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08b9aa-bad0-448c-afd6-c3c79c90c505 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.425004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08b9aa-bad0-448c-afd6-c3c79c90c505 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.425004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08b9aa-bad0-448c-afd6-c3c79c90c505 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.425399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 234/473] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:04.426123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08b9aa-bad0-448c-afd6-c3c79c90c505 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:04.561359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cb7a0-7f0f-48fb-b7a1-459988c1fc07 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:04.567564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08b9aa-bad0-448c-afd6-c3c79c90c505 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:04.585155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:04.589795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:04.611004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc4b51-ba44-4a6b-ac56-85fabd32a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:04.611808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d778c28-b642-4282-a331-e3c4600cc53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:04.612105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc4b51-ba44-4a6b-ac56-85fabd32a474 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.612530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc4b51-ba44-4a6b-ac56-85fabd32a474 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.612848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d778c28-b642-4282-a331-e3c4600cc53e None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.613116 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc4b51-ba44-4a6b-ac56-85fabd32a474 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.613223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d778c28-b642-4282-a331-e3c4600cc53e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.613405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d778c28-b642-4282-a331-e3c4600cc53e None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.614111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d778c28-b642-4282-a331-e3c4600cc53e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:04.614366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc4b51-ba44-4a6b-ac56-85fabd32a474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:04.615539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc4b51-ba44-4a6b-ac56-85fabd32a474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:04.615648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d778c28-b642-4282-a331-e3c4600cc53e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:04.617108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc4b51-ba44-4a6b-ac56-85fabd32a474 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:04.633538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 235/474] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:04.635294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846820-d446-47b7-8976-622ea83f6829 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.635834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846820-d446-47b7-8976-622ea83f6829 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.636295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846820-d446-47b7-8976-622ea83f6829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.642968 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 240/475] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:37:04 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:04.644014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d778c28-b642-4282-a331-e3c4600cc53e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:04.659414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-86846820-d446-47b7-8976-622ea83f6829 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:04.660685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-86846820-d446-47b7-8976-622ea83f6829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:04.748435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 236/476] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:04.777524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86846820-d446-47b7-8976-622ea83f6829 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:04.783877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d778c28-b642-4282-a331-e3c4600cc53e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:04.826579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e5985-fe76-4098-b177-14ec3b913ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:04.827652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e5985-fe76-4098-b177-14ec3b913ef9 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.828132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e5985-fe76-4098-b177-14ec3b913ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.828651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e5985-fe76-4098-b177-14ec3b913ef9 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.829710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e5985-fe76-4098-b177-14ec3b913ef9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:04.831557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e5985-fe76-4098-b177-14ec3b913ef9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:04.870905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 241/477] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:37:04 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:04.890456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86846820-d446-47b7-8976-622ea83f6829 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:04.893376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:04.930915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:04.931621 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:04.931909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:04.932197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:04.932976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:04.934420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:04.942386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.sql.core [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 15:37:04.942386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 15:37:04.942386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [parameters: {'id': '15362fd61b9a4b69a2163f6278a19bb4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 15:37:04.942386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86639) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application conn.query(q) Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result.read() Jun 15 15:37:04.959524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return ref.to_dict() Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 15:37:04.960441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application next(self.gen) Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application yield resource Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application next(self.gen) Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise self.value Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application session.commit() Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 15:37:04.962281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.session.flush() Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application compat.raise_( Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 15:37:04.963158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 15:37:04.964081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application conn.query(q) Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result.read() Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 15:37:04.964861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [parameters: {'id': '15362fd61b9a4b69a2163f6278a19bb4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:04.965638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 15:37:04.966426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:04.967289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 237/478] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Jun 15 15:37:04 2022] POST /identity/v3/roles => generated 144 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:04.970029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e5985-fe76-4098-b177-14ec3b913ef9 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:05.011455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:05.095118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e5985-fe76-4098-b177-14ec3b913ef9 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:05.130506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206883d-f69b-4167-bb52-562889cc46d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:05.131357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206883d-f69b-4167-bb52-562889cc46d9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:05.131642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206883d-f69b-4167-bb52-562889cc46d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:05.131921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206883d-f69b-4167-bb52-562889cc46d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:05.144014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206883d-f69b-4167-bb52-562889cc46d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-3206883d-f69b-4167-bb52-562889cc46d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:05.153086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:05.154014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:05.154843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 242/479] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:05.160480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:05.276966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449c5d2-f643-4597-911f-6f6d9ee949b8 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:05.313322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b0f6d-36f9-4acc-915d-65a5105fe575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:05.313998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b0f6d-36f9-4acc-915d-65a5105fe575 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:05.314283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b0f6d-36f9-4acc-915d-65a5105fe575 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:05.314563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b0f6d-36f9-4acc-915d-65a5105fe575 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:05.315357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b0f6d-36f9-4acc-915d-65a5105fe575 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:05.315670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b0f6d-36f9-4acc-915d-65a5105fe575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:05.317029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b0f6d-36f9-4acc-915d-65a5105fe575 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:05.326570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 238/480] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:06.951027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a36a3-b0c0-4f52-be85-bb44630068d0 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:06.952030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a36a3-b0c0-4f52-be85-bb44630068d0 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:06.952523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a36a3-b0c0-4f52-be85-bb44630068d0 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:06.954344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 243/481] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:06.959929 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd4e14-2d7b-4371-a075-567f1eea2f8b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:06.960762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd4e14-2d7b-4371-a075-567f1eea2f8b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:06.961825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd4e14-2d7b-4371-a075-567f1eea2f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:06.980685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-92bd4e14-2d7b-4371-a075-567f1eea2f8b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:06.982302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-92bd4e14-2d7b-4371-a075-567f1eea2f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:07.011325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:07.055797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa26298-3f64-468f-b926-469917363da0 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.056724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa26298-3f64-468f-b926-469917363da0 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.057474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa26298-3f64-468f-b926-469917363da0 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.059336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 244/482] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:07.060790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 239/483] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:07.065729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383c3e-1907-4089-9e1b-15704271913f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.066242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383c3e-1907-4089-9e1b-15704271913f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.066723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383c3e-1907-4089-9e1b-15704271913f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.097152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-86383c3e-1907-4089-9e1b-15704271913f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:07.098652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-86383c3e-1907-4089-9e1b-15704271913f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:07.124580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd4e14-2d7b-4371-a075-567f1eea2f8b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:07.189819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 245/484] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:07.238559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9021c0-b29d-4bf9-b770-59d51b91627e None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.238766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9021c0-b29d-4bf9-b770-59d51b91627e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.239066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9021c0-b29d-4bf9-b770-59d51b91627e None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.240610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 246/485] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:07.246163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9021c0-b29d-4bf9-b770-59d51b91627e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:07.266215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:07.291355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd4e14-2d7b-4371-a075-567f1eea2f8b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:07.325420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70084b7-e96a-43cf-8ba8-b046cf8e34e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:07.326228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70084b7-e96a-43cf-8ba8-b046cf8e34e7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.326514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70084b7-e96a-43cf-8ba8-b046cf8e34e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.326822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70084b7-e96a-43cf-8ba8-b046cf8e34e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.337911 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70084b7-e96a-43cf-8ba8-b046cf8e34e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-c70084b7-e96a-43cf-8ba8-b046cf8e34e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:07.345670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:07.346575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:07.347414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:07.350805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 240/486] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:07.352644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee180d-b26b-495a-a45f-4a22064cbab1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.352930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee180d-b26b-495a-a45f-4a22064cbab1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.353209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee180d-b26b-495a-a45f-4a22064cbab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.369837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9021c0-b29d-4bf9-b770-59d51b91627e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:07.371230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ecee180d-b26b-495a-a45f-4a22064cbab1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:07.372455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ecee180d-b26b-495a-a45f-4a22064cbab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:07.387127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:07.413736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726ab27-5838-43a6-ab77-88a8d7de3d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:07.415029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726ab27-5838-43a6-ab77-88a8d7de3d47 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.415315 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726ab27-5838-43a6-ab77-88a8d7de3d47 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.415589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726ab27-5838-43a6-ab77-88a8d7de3d47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.430314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726ab27-5838-43a6-ab77-88a8d7de3d47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-d726ab27-5838-43a6-ab77-88a8d7de3d47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:07.438029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:07.439157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:07.440156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:07.442969 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 247/487] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:07.443783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726ab27-5838-43a6-ab77-88a8d7de3d47 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:07.451432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 241/488] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:07.452466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee180d-b26b-495a-a45f-4a22064cbab1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:07.545333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:07.572048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726ab27-5838-43a6-ab77-88a8d7de3d47 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:07.578053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee180d-b26b-495a-a45f-4a22064cbab1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:07.607338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf94eee-2735-4223-baa0-ef843dcf8b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:07.608866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf94eee-2735-4223-baa0-ef843dcf8b53 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.609358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf94eee-2735-4223-baa0-ef843dcf8b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.610001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf94eee-2735-4223-baa0-ef843dcf8b53 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.611542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf94eee-2735-4223-baa0-ef843dcf8b53 None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:07.612095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf94eee-2735-4223-baa0-ef843dcf8b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:07.613697 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf94eee-2735-4223-baa0-ef843dcf8b53 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:07.615407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85762747-7fc9-413c-bb33-c2c59d6d4b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:07.616411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762747-7fc9-413c-bb33-c2c59d6d4b80 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.616916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762747-7fc9-413c-bb33-c2c59d6d4b80 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.617974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762747-7fc9-413c-bb33-c2c59d6d4b80 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.618957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85762747-7fc9-413c-bb33-c2c59d6d4b80 None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:07.619481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85762747-7fc9-413c-bb33-c2c59d6d4b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:07.621126 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85762747-7fc9-413c-bb33-c2c59d6d4b80 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:07.623361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 248/489] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:07.624481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf94eee-2735-4223-baa0-ef843dcf8b53 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:07.631215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 242/490] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:07.632324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85762747-7fc9-413c-bb33-c2c59d6d4b80 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:07.649780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:07.741407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf94eee-2735-4223-baa0-ef843dcf8b53 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:07.760323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85762747-7fc9-413c-bb33-c2c59d6d4b80 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:07.780401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f7c18-e5c6-4cf4-a873-dc42651bead4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:07.781435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f7c18-e5c6-4cf4-a873-dc42651bead4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.781881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f7c18-e5c6-4cf4-a873-dc42651bead4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.782545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f7c18-e5c6-4cf4-a873-dc42651bead4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.795732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:07.803319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f7c18-e5c6-4cf4-a873-dc42651bead4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:07.806185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbec1b-4009-4e7f-af53-66a7cf45b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:07.807349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbec1b-4009-4e7f-af53-66a7cf45b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.807814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbec1b-4009-4e7f-af53-66a7cf45b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.808263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbec1b-4009-4e7f-af53-66a7cf45b8d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-fd6f7c18-e5c6-4cf4-a873-dc42651bead4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:07.813348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:07.814279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:07.815142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:07.818876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 249/491] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:07.819910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f7c18-e5c6-4cf4-a873-dc42651bead4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:07.824717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbec1b-4009-4e7f-af53-66a7cf45b8d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:07.827420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbec1b-4009-4e7f-af53-66a7cf45b8d6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-05dbec1b-4009-4e7f-af53-66a7cf45b8d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:07.853468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:07.854357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:07.855247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:07.855247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:07.855247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:07.855247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:07.856225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 243/492] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:07.856986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbec1b-4009-4e7f-af53-66a7cf45b8d6 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:07.925980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:07.939650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f7c18-e5c6-4cf4-a873-dc42651bead4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:07.969381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbec1b-4009-4e7f-af53-66a7cf45b8d6 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:07.974837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97cead-c6a2-4aee-bcac-d2d8a0eff029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:07.975626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97cead-c6a2-4aee-bcac-d2d8a0eff029 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:07.975910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97cead-c6a2-4aee-bcac-d2d8a0eff029 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:07.976184 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97cead-c6a2-4aee-bcac-d2d8a0eff029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:07.986525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97cead-c6a2-4aee-bcac-d2d8a0eff029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:07.988031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97cead-c6a2-4aee-bcac-d2d8a0eff029 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.005638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17ba88-a6d1-416a-a2f4-30c04820a81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.006308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17ba88-a6d1-416a-a2f4-30c04820a81a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.006592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17ba88-a6d1-416a-a2f4-30c04820a81a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.006901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17ba88-a6d1-416a-a2f4-30c04820a81a None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.007695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17ba88-a6d1-416a-a2f4-30c04820a81a None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:08.008009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17ba88-a6d1-416a-a2f4-30c04820a81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-7d97cead-c6a2-4aee-bcac-d2d8a0eff029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:08.008624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:08.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:08.010666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:08.010666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:08.010666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:08.010666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:08.010964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17ba88-a6d1-416a-a2f4-30c04820a81a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.012323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 250/493] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:08.013721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762248a-ff70-4ee1-9eba-f7a8b2086b0f None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.014138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762248a-ff70-4ee1-9eba-f7a8b2086b0f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.014542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762248a-ff70-4ee1-9eba-f7a8b2086b0f None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.015966 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 251/494] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:08.016801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762248a-ff70-4ee1-9eba-f7a8b2086b0f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.024309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 244/495] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:08.025137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17ba88-a6d1-416a-a2f4-30c04820a81a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.080134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:08.129868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17ba88-a6d1-416a-a2f4-30c04820a81a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.142402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762248a-ff70-4ee1-9eba-f7a8b2086b0f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.157193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:08.167627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3717-7968-4e53-bc9f-e06f06fa61be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.168288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3717-7968-4e53-bc9f-e06f06fa61be None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.168568 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3717-7968-4e53-bc9f-e06f06fa61be None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.168842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3717-7968-4e53-bc9f-e06f06fa61be None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.169563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3717-7968-4e53-bc9f-e06f06fa61be None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:08.169872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3717-7968-4e53-bc9f-e06f06fa61be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.171264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3717-7968-4e53-bc9f-e06f06fa61be None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.181602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787159ba-d0cd-4695-9dce-e16ebf4ba84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.182599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787159ba-d0cd-4695-9dce-e16ebf4ba84f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.183101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787159ba-d0cd-4695-9dce-e16ebf4ba84f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.183526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787159ba-d0cd-4695-9dce-e16ebf4ba84f None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.183628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 245/496] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:08.184458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787159ba-d0cd-4695-9dce-e16ebf4ba84f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:08.184838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55110d47-2bb1-441b-b4b9-58a65f4b8108 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.185055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787159ba-d0cd-4695-9dce-e16ebf4ba84f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.185153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55110d47-2bb1-441b-b4b9-58a65f4b8108 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.185388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55110d47-2bb1-441b-b4b9-58a65f4b8108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.186844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787159ba-d0cd-4695-9dce-e16ebf4ba84f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.201501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 252/497] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:08.202451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787159ba-d0cd-4695-9dce-e16ebf4ba84f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.203348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-55110d47-2bb1-441b-b4b9-58a65f4b8108 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:08.204681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-55110d47-2bb1-441b-b4b9-58a65f4b8108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:08.286420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 246/498] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:08.287298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55110d47-2bb1-441b-b4b9-58a65f4b8108 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.319185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787159ba-d0cd-4695-9dce-e16ebf4ba84f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.334334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:08.356222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bba1c-10ac-4a8a-bf22-50079080e303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.357030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bba1c-10ac-4a8a-bf22-50079080e303 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.357311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bba1c-10ac-4a8a-bf22-50079080e303 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.357586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bba1c-10ac-4a8a-bf22-50079080e303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.368116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bba1c-10ac-4a8a-bf22-50079080e303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.369585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bba1c-10ac-4a8a-bf22-50079080e303 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-cf1bba1c-10ac-4a8a-bf22-50079080e303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:08.387057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:08.387896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:08.388736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:08.388736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:08.388736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:08.388736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:08.388736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 253/499] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:08.389009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bba1c-10ac-4a8a-bf22-50079080e303 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.396692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55110d47-2bb1-441b-b4b9-58a65f4b8108 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.414121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:08.432194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb3477-5c57-4e55-babe-d14db54bc0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.433118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb3477-5c57-4e55-babe-d14db54bc0ad None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.433411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb3477-5c57-4e55-babe-d14db54bc0ad None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.433914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb3477-5c57-4e55-babe-d14db54bc0ad None admin] PATH_INFO: `/v3/users/nova` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.442651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb3477-5c57-4e55-babe-d14db54bc0ad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.444137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb3477-5c57-4e55-babe-d14db54bc0ad None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-d1bb3477-5c57-4e55-babe-d14db54bc0ad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:08.463360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:08.464428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:08.465356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:08.466724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 247/500] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:08.468004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fbbb9-3872-4c1e-af01-bd17d8a0c9fb None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.468299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fbbb9-3872-4c1e-af01-bd17d8a0c9fb None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.468579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fbbb9-3872-4c1e-af01-bd17d8a0c9fb None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.470597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 248/501] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:08.471320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fbbb9-3872-4c1e-af01-bd17d8a0c9fb None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.493068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bba1c-10ac-4a8a-bf22-50079080e303 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.526278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f3a77-a3c6-4d36-b557-e7820190bc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.527098 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f3a77-a3c6-4d36-b557-e7820190bc5d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.527374 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f3a77-a3c6-4d36-b557-e7820190bc5d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.527650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f3a77-a3c6-4d36-b557-e7820190bc5d None admin] PATH_INFO: `/v3/users/glance` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.534667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f3a77-a3c6-4d36-b557-e7820190bc5d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.536135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f3a77-a3c6-4d36-b557-e7820190bc5d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-a73f3a77-a3c6-4d36-b557-e7820190bc5d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:08.551928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:08.552939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:08.553838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 254/502] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:08.554400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f3a77-a3c6-4d36-b557-e7820190bc5d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.575387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fbbb9-3872-4c1e-af01-bd17d8a0c9fb None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.578048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:08.610070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f47f4c-5017-4f51-a4ca-fe4b10ad8f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.610818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f47f4c-5017-4f51-a4ca-fe4b10ad8f86 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.611107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f47f4c-5017-4f51-a4ca-fe4b10ad8f86 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.611381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f47f4c-5017-4f51-a4ca-fe4b10ad8f86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.621789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f47f4c-5017-4f51-a4ca-fe4b10ad8f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-a3f47f4c-5017-4f51-a4ca-fe4b10ad8f86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:08.629456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:08.630323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 249/503] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:08.631486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f47f4c-5017-4f51-a4ca-fe4b10ad8f86 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.653759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f3a77-a3c6-4d36-b557-e7820190bc5d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.670074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:08.689238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbce51-6828-4613-bf98-3ce783d0d7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.689898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbce51-6828-4613-bf98-3ce783d0d7a6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.690186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbce51-6828-4613-bf98-3ce783d0d7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.690461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbce51-6828-4613-bf98-3ce783d0d7a6 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.691215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbce51-6828-4613-bf98-3ce783d0d7a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:08.691520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbce51-6828-4613-bf98-3ce783d0d7a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.692910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbce51-6828-4613-bf98-3ce783d0d7a6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.704347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 255/504] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:08.705519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddf26d-5dd1-4017-935d-677e985bcfaf None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.705844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddf26d-5dd1-4017-935d-677e985bcfaf None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.706126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddf26d-5dd1-4017-935d-677e985bcfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.722163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2eddf26d-5dd1-4017-935d-677e985bcfaf None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:08.723378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2eddf26d-5dd1-4017-935d-677e985bcfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:08.736460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f47f4c-5017-4f51-a4ca-fe4b10ad8f86 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.768737 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416ac96-2f27-461c-9b76-44b121f388ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.769392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416ac96-2f27-461c-9b76-44b121f388ee None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.769673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416ac96-2f27-461c-9b76-44b121f388ee None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.769948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416ac96-2f27-461c-9b76-44b121f388ee None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.770811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416ac96-2f27-461c-9b76-44b121f388ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:08.771379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416ac96-2f27-461c-9b76-44b121f388ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.772812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416ac96-2f27-461c-9b76-44b121f388ee None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.789474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 256/505] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:08.790284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddf26d-5dd1-4017-935d-677e985bcfaf None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.812270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 250/506] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:08.813206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416ac96-2f27-461c-9b76-44b121f388ee None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.820171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:08.903363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddf26d-5dd1-4017-935d-677e985bcfaf None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.904765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:08.911764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416ac96-2f27-461c-9b76-44b121f388ee None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:08.937332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd446aa7-0b5c-4aa7-8387-118c7f8c630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.937992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd446aa7-0b5c-4aa7-8387-118c7f8c630b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.938274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd446aa7-0b5c-4aa7-8387-118c7f8c630b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.938549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd446aa7-0b5c-4aa7-8387-118c7f8c630b None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.939413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd446aa7-0b5c-4aa7-8387-118c7f8c630b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:08.939719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd446aa7-0b5c-4aa7-8387-118c7f8c630b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.941166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd446aa7-0b5c-4aa7-8387-118c7f8c630b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.944545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0746c03-b425-4201-960c-0d20ee718e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:08.945758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0746c03-b425-4201-960c-0d20ee718e7d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:08.946041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0746c03-b425-4201-960c-0d20ee718e7d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:08.946317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0746c03-b425-4201-960c-0d20ee718e7d None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:08.947093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0746c03-b425-4201-960c-0d20ee718e7d None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:08.947404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0746c03-b425-4201-960c-0d20ee718e7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:08.948758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0746c03-b425-4201-960c-0d20ee718e7d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:08.956578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 251/507] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:08.957355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0746c03-b425-4201-960c-0d20ee718e7d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:08.977651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 257/508] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:08.978489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd446aa7-0b5c-4aa7-8387-118c7f8c630b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.055619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0746c03-b425-4201-960c-0d20ee718e7d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.058452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:09.087034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd446aa7-0b5c-4aa7-8387-118c7f8c630b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.088625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3f1d2-10e1-49b3-a034-d33d379ff52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.089414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3f1d2-10e1-49b3-a034-d33d379ff52e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.089698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3f1d2-10e1-49b3-a034-d33d379ff52e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.089974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3f1d2-10e1-49b3-a034-d33d379ff52e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.097243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3f1d2-10e1-49b3-a034-d33d379ff52e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.098691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3f1d2-10e1-49b3-a034-d33d379ff52e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-04e3f1d2-10e1-49b3-a034-d33d379ff52e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:09.113085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:09.114162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 252/509] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:09.115118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3f1d2-10e1-49b3-a034-d33d379ff52e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.124020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb15a-a9b5-4670-ae3d-d274716ad4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.124807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb15a-a9b5-4670-ae3d-d274716ad4b8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.125089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb15a-a9b5-4670-ae3d-d274716ad4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.125364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb15a-a9b5-4670-ae3d-d274716ad4b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.132663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:09.135901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb15a-a9b5-4670-ae3d-d274716ad4b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.137420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb15a-a9b5-4670-ae3d-d274716ad4b8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-878bb15a-a9b5-4670-ae3d-d274716ad4b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:09.155268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:09.156169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:09.157211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:09.157211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:09.157211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:09.157211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:09.157211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 258/510] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:09.157645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb15a-a9b5-4670-ae3d-d274716ad4b8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.219619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3f1d2-10e1-49b3-a034-d33d379ff52e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.253388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6f7d4-e0d8-4ad5-8966-9e5af22c6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.254026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6f7d4-e0d8-4ad5-8966-9e5af22c6e14 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.254165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb15a-a9b5-4670-ae3d-d274716ad4b8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.254311 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6f7d4-e0d8-4ad5-8966-9e5af22c6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.254576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6f7d4-e0d8-4ad5-8966-9e5af22c6e14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.277618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:09.288908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db10cb4-25c7-44c9-8830-fa9883817d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.289697 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db10cb4-25c7-44c9-8830-fa9883817d27 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.289985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db10cb4-25c7-44c9-8830-fa9883817d27 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.290259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db10cb4-25c7-44c9-8830-fa9883817d27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.300103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db10cb4-25c7-44c9-8830-fa9883817d27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.301568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db10cb4-25c7-44c9-8830-fa9883817d27 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-6db10cb4-25c7-44c9-8830-fa9883817d27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:09.319277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:09.320203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:09.321061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:09.321061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:09.321061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:09.321061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:09.321061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 259/511] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:09.321319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db10cb4-25c7-44c9-8830-fa9883817d27 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.326164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6f7d4-e0d8-4ad5-8966-9e5af22c6e14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.327680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6f7d4-e0d8-4ad5-8966-9e5af22c6e14 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.345836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:09.419465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db10cb4-25c7-44c9-8830-fa9883817d27 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.427063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 253/512] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:09.427874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6f7d4-e0d8-4ad5-8966-9e5af22c6e14 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.455681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83ac51-36b4-4447-ab17-37d764f4328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.456319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83ac51-36b4-4447-ab17-37d764f4328f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.456597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83ac51-36b4-4447-ab17-37d764f4328f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.456872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83ac51-36b4-4447-ab17-37d764f4328f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.501731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:09.522973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6f7d4-e0d8-4ad5-8966-9e5af22c6e14 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.524979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83ac51-36b4-4447-ab17-37d764f4328f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.526458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83ac51-36b4-4447-ab17-37d764f4328f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.539862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:09.558396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14813dc-8a4e-4a72-b338-33d5ddd7a7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.559081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14813dc-8a4e-4a72-b338-33d5ddd7a7a3 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.559360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14813dc-8a4e-4a72-b338-33d5ddd7a7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.559631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14813dc-8a4e-4a72-b338-33d5ddd7a7a3 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.560456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14813dc-8a4e-4a72-b338-33d5ddd7a7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:09.560767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14813dc-8a4e-4a72-b338-33d5ddd7a7a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.562185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14813dc-8a4e-4a72-b338-33d5ddd7a7a3 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.578625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 254/513] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Jun 15 15:37:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:09.579435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14813dc-8a4e-4a72-b338-33d5ddd7a7a3 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.626742 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 260/514] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:09.627614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83ac51-36b4-4447-ab17-37d764f4328f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.681254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14813dc-8a4e-4a72-b338-33d5ddd7a7a3 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.701190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:09.714748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e9b1e-9935-4ce3-94f7-689bda040ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.715416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e9b1e-9935-4ce3-94f7-689bda040ddb None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.715702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e9b1e-9935-4ce3-94f7-689bda040ddb None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.715981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e9b1e-9935-4ce3-94f7-689bda040ddb None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.716750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e9b1e-9935-4ce3-94f7-689bda040ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:09.717007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e9b1e-9935-4ce3-94f7-689bda040ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.718357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e9b1e-9935-4ce3-94f7-689bda040ddb None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.723801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83ac51-36b4-4447-ab17-37d764f4328f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.729907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 255/515] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:09.730522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e9b1e-9935-4ce3-94f7-689bda040ddb None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.756551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb847525-1c5c-4c59-af41-c9a33ad6b6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.757206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb847525-1c5c-4c59-af41-c9a33ad6b6de None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.758080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb847525-1c5c-4c59-af41-c9a33ad6b6de None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.758360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb847525-1c5c-4c59-af41-c9a33ad6b6de None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.759099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb847525-1c5c-4c59-af41-c9a33ad6b6de None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:09.759403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb847525-1c5c-4c59-af41-c9a33ad6b6de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.760778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb847525-1c5c-4c59-af41-c9a33ad6b6de None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.768037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:09.772667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 261/516] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:09.773495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb847525-1c5c-4c59-af41-c9a33ad6b6de None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.835258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e9b1e-9935-4ce3-94f7-689bda040ddb None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.868800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474d1ca-94a9-41fe-9c04-407f7c1fa6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.869577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474d1ca-94a9-41fe-9c04-407f7c1fa6e0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.869858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474d1ca-94a9-41fe-9c04-407f7c1fa6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.870135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474d1ca-94a9-41fe-9c04-407f7c1fa6e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.871806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb847525-1c5c-4c59-af41-c9a33ad6b6de None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:09.880393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474d1ca-94a9-41fe-9c04-407f7c1fa6e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.881828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474d1ca-94a9-41fe-9c04-407f7c1fa6e0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-3474d1ca-94a9-41fe-9c04-407f7c1fa6e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:09.899623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:09.900632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:09.901709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:09.901709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:09.901709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:09.901709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:09.902876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 256/517] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:09.903828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474d1ca-94a9-41fe-9c04-407f7c1fa6e0 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:09.905496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eba95a-a570-4c24-9533-a1e1a18dfdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:09.906137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eba95a-a570-4c24-9533-a1e1a18dfdfe None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:09.906418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eba95a-a570-4c24-9533-a1e1a18dfdfe None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:09.906691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eba95a-a570-4c24-9533-a1e1a18dfdfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:09.910086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:09.974062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eba95a-a570-4c24-9533-a1e1a18dfdfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:09.975477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eba95a-a570-4c24-9533-a1e1a18dfdfe None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:09.987574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:10.002503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474d1ca-94a9-41fe-9c04-407f7c1fa6e0 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.036979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395301b7-b9ae-4745-a513-fbca2a9e4e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.037761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395301b7-b9ae-4745-a513-fbca2a9e4e77 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.038047 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395301b7-b9ae-4745-a513-fbca2a9e4e77 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.038325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395301b7-b9ae-4745-a513-fbca2a9e4e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.049749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395301b7-b9ae-4745-a513-fbca2a9e4e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.051246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395301b7-b9ae-4745-a513-fbca2a9e4e77 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-395301b7-b9ae-4745-a513-fbca2a9e4e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:10.069102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:10.070488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:10.071537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:10.071537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:10.071537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:10.071537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:10.071537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 257/518] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:10.071537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395301b7-b9ae-4745-a513-fbca2a9e4e77 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.082333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 262/519] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:10.082333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eba95a-a570-4c24-9533-a1e1a18dfdfe None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.112801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:10.169744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395301b7-b9ae-4745-a513-fbca2a9e4e77 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.177858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eba95a-a570-4c24-9533-a1e1a18dfdfe None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.204087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784a28-c41d-4293-93d1-c0c3c5319412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.204741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784a28-c41d-4293-93d1-c0c3c5319412 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.205022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784a28-c41d-4293-93d1-c0c3c5319412 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.205295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784a28-c41d-4293-93d1-c0c3c5319412 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.205988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10784a28-c41d-4293-93d1-c0c3c5319412 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.207461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10784a28-c41d-4293-93d1-c0c3c5319412 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.211927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685f8a-a5bc-495c-a5e8-9314664c2f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.212746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685f8a-a5bc-495c-a5e8-9314664c2f17 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.213028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685f8a-a5bc-495c-a5e8-9314664c2f17 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.213322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685f8a-a5bc-495c-a5e8-9314664c2f17 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.216692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:10.220823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a685f8a-a5bc-495c-a5e8-9314664c2f17 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.222510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a685f8a-a5bc-495c-a5e8-9314664c2f17 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.229510 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 258/520] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:37:10 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:10.230232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784a28-c41d-4293-93d1-c0c3c5319412 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-3a685f8a-a5bc-495c-a5e8-9314664c2f17 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:10.237638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:10.238660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:10.239664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 263/521] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:10.240343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685f8a-a5bc-495c-a5e8-9314664c2f17 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.354580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:10.360054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784a28-c41d-4293-93d1-c0c3c5319412 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.361799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685f8a-a5bc-495c-a5e8-9314664c2f17 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.397068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6b756-f514-43c0-ac5a-477835716ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.397738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6b756-f514-43c0-ac5a-477835716ec8 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.398049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6b756-f514-43c0-ac5a-477835716ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.398875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6b756-f514-43c0-ac5a-477835716ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.399604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6b756-f514-43c0-ac5a-477835716ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:10.399914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6b756-f514-43c0-ac5a-477835716ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.401282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6b756-f514-43c0-ac5a-477835716ec8 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.404285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d72fa-b701-4911-a229-c3955de1ce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.405027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d72fa-b701-4911-a229-c3955de1ce1f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.405414 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d72fa-b701-4911-a229-c3955de1ce1f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.405762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d72fa-b701-4911-a229-c3955de1ce1f None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.406568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d72fa-b701-4911-a229-c3955de1ce1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:10.406976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d72fa-b701-4911-a229-c3955de1ce1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.408527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d72fa-b701-4911-a229-c3955de1ce1f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.413614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 259/522] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:10.414493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6b756-f514-43c0-ac5a-477835716ec8 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.422698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 264/523] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:10.423750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d72fa-b701-4911-a229-c3955de1ce1f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.481522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:10.529931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6b756-f514-43c0-ac5a-477835716ec8 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.540548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d72fa-b701-4911-a229-c3955de1ce1f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.564863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd08c0e-29be-4356-9ba7-4fc67795de66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.565649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd08c0e-29be-4356-9ba7-4fc67795de66 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.565935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd08c0e-29be-4356-9ba7-4fc67795de66 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.566216 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd08c0e-29be-4356-9ba7-4fc67795de66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.575602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c9b70-eabd-4a5f-a414-e122e4660b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.576264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c9b70-eabd-4a5f-a414-e122e4660b72 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.576545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c9b70-eabd-4a5f-a414-e122e4660b72 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.576807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd08c0e-29be-4356-9ba7-4fc67795de66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.576917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c9b70-eabd-4a5f-a414-e122e4660b72 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.577634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c9b70-eabd-4a5f-a414-e122e4660b72 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:10.577950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c9b70-eabd-4a5f-a414-e122e4660b72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.578261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd08c0e-29be-4356-9ba7-4fc67795de66 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.579581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c9b70-eabd-4a5f-a414-e122e4660b72 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-2dd08c0e-29be-4356-9ba7-4fc67795de66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:10.596590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:10.597493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:10.599076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:10.599076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:10.599076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:10.599076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:10.599313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 265/524] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:10.599313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c9b70-eabd-4a5f-a414-e122e4660b72 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.602822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 260/525] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:10.603698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd08c0e-29be-4356-9ba7-4fc67795de66 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.619289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:10.709520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd08c0e-29be-4356-9ba7-4fc67795de66 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.711030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c9b70-eabd-4a5f-a414-e122e4660b72 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.717480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:10.748075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baece3f6-ea25-4c1f-b2e1-958ae1ba0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.748532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b0add-e2e7-459e-842a-6385086d0e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.748788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baece3f6-ea25-4c1f-b2e1-958ae1ba0316 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.749072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baece3f6-ea25-4c1f-b2e1-958ae1ba0316 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.749328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b0add-e2e7-459e-842a-6385086d0e59 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.749424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baece3f6-ea25-4c1f-b2e1-958ae1ba0316 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.749614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b0add-e2e7-459e-842a-6385086d0e59 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.749894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b0add-e2e7-459e-842a-6385086d0e59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.757235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b0add-e2e7-459e-842a-6385086d0e59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.758810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b0add-e2e7-459e-842a-6385086d0e59 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.760138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baece3f6-ea25-4c1f-b2e1-958ae1ba0316 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.761812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baece3f6-ea25-4c1f-b2e1-958ae1ba0316 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-6a5b0add-e2e7-459e-842a-6385086d0e59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:10.773239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:10.774070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:10.775288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:10.775288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:10.775288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:10.775288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:10.775288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:10.775288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:10.775288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:10.775288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-baece3f6-ea25-4c1f-b2e1-958ae1ba0316 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:10.776549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:10.777406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:10.778244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:10.778244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:10.778244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:10.778244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:10.778244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:10.778244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:10.778244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:10.778244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:10.779469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 261/526] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:10.780248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b0add-e2e7-459e-842a-6385086d0e59 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.782231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 266/527] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:10.783072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baece3f6-ea25-4c1f-b2e1-958ae1ba0316 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.869998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:10.889604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b0add-e2e7-459e-842a-6385086d0e59 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.895528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baece3f6-ea25-4c1f-b2e1-958ae1ba0316 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:10.930947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bea07-f9bc-4e6d-9e04-bbe5202661b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.931607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bea07-f9bc-4e6d-9e04-bbe5202661b2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.931893 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bea07-f9bc-4e6d-9e04-bbe5202661b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.932171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bea07-f9bc-4e6d-9e04-bbe5202661b2 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.932889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bea07-f9bc-4e6d-9e04-bbe5202661b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:10.933203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bea07-f9bc-4e6d-9e04-bbe5202661b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:10.933307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec89c3d-e4ca-4846-be1f-cdc7aff1c108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:10.933920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec89c3d-e4ca-4846-be1f-cdc7aff1c108 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:10.934205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec89c3d-e4ca-4846-be1f-cdc7aff1c108 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:10.934484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec89c3d-e4ca-4846-be1f-cdc7aff1c108 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:10.934581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bea07-f9bc-4e6d-9e04-bbe5202661b2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:10.951191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 267/528] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:10.951191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bea07-f9bc-4e6d-9e04-bbe5202661b2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:10.980842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:11.021623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec89c3d-e4ca-4846-be1f-cdc7aff1c108 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.023572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec89c3d-e4ca-4846-be1f-cdc7aff1c108 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:11.050156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bea07-f9bc-4e6d-9e04-bbe5202661b2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:11.087911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f33af9-aa01-45d9-9bdf-b9e358f3b454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:11.088581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f33af9-aa01-45d9-9bdf-b9e358f3b454 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:11.088864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f33af9-aa01-45d9-9bdf-b9e358f3b454 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:11.089168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f33af9-aa01-45d9-9bdf-b9e358f3b454 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:11.090469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f33af9-aa01-45d9-9bdf-b9e358f3b454 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:11.091018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f33af9-aa01-45d9-9bdf-b9e358f3b454 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.093341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f33af9-aa01-45d9-9bdf-b9e358f3b454 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:11.095240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:11.120953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 268/529] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:11.121800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f33af9-aa01-45d9-9bdf-b9e358f3b454 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:11.138175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 262/530] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:11.139709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec89c3d-e4ca-4846-be1f-cdc7aff1c108 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:11.207955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:11.233515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f33af9-aa01-45d9-9bdf-b9e358f3b454 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:11.241301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec89c3d-e4ca-4846-be1f-cdc7aff1c108 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:11.271303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9147e6a-f580-4410-a7e9-c8a2b9629286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:11.272506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9147e6a-f580-4410-a7e9-c8a2b9629286 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:11.273066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9147e6a-f580-4410-a7e9-c8a2b9629286 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:11.273526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9147e6a-f580-4410-a7e9-c8a2b9629286 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:11.274576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f516e0-83b8-43ed-a220-8f732e3ce362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:11.274810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9147e6a-f580-4410-a7e9-c8a2b9629286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:11.275308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9147e6a-f580-4410-a7e9-c8a2b9629286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.275410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f516e0-83b8-43ed-a220-8f732e3ce362 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:11.275650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f516e0-83b8-43ed-a220-8f732e3ce362 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:11.275926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f516e0-83b8-43ed-a220-8f732e3ce362 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:11.277692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9147e6a-f580-4410-a7e9-c8a2b9629286 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:11.282412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f516e0-83b8-43ed-a220-8f732e3ce362 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.283859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f516e0-83b8-43ed-a220-8f732e3ce362 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:11.291175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 269/531] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:11.292219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9147e6a-f580-4410-a7e9-c8a2b9629286 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-43f516e0-83b8-43ed-a220-8f732e3ce362 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:11.297886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:11.298761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:11.299656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:11.299656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:11.299656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:11.299656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:11.299656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:11.299656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:11.299656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:11.299656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:11.303637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 263/532] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:11.304301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f516e0-83b8-43ed-a220-8f732e3ce362 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:11.355595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:11.407296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f516e0-83b8-43ed-a220-8f732e3ce362 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:11.408465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9147e6a-f580-4410-a7e9-c8a2b9629286 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:11.438310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:11.447086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbb247-90fd-424b-9526-4981f3c60325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:11.447737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbb247-90fd-424b-9526-4981f3c60325 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:11.448086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbb247-90fd-424b-9526-4981f3c60325 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:11.448375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbb247-90fd-424b-9526-4981f3c60325 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:11.449224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbb247-90fd-424b-9526-4981f3c60325 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=5f7405c8bc8e4c84bb056cd9e1df61c7) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:11.449563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbb247-90fd-424b-9526-4981f3c60325 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.449704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98788731-2b17-4f75-810f-23cac6d830e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:11.450644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98788731-2b17-4f75-810f-23cac6d830e7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:11.450998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbb247-90fd-424b-9526-4981f3c60325 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:11.451265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98788731-2b17-4f75-810f-23cac6d830e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:11.451683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98788731-2b17-4f75-810f-23cac6d830e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:11.457095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 270/533] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:11 2022] GET /identity/v3/role_assignments?user.id=5f7405c8bc8e4c84bb056cd9e1df61c7&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:11.457892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbb247-90fd-424b-9526-4981f3c60325 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:11.464143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98788731-2b17-4f75-810f-23cac6d830e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.465641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98788731-2b17-4f75-810f-23cac6d830e7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-98788731-2b17-4f75-810f-23cac6d830e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:11.487161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:11.488275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:11.489366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:11.489366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:11.489366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:11.489366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:11.489366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 264/534] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:11.489366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98788731-2b17-4f75-810f-23cac6d830e7 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:11.610469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98788731-2b17-4f75-810f-23cac6d830e7 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:11.623519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbb247-90fd-424b-9526-4981f3c60325 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:11.649153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:11.652749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04673666-6532-4465-8b91-25ad9a5f5e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:11.653833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04673666-6532-4465-8b91-25ad9a5f5e1c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:11.654407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04673666-6532-4465-8b91-25ad9a5f5e1c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:11.654917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04673666-6532-4465-8b91-25ad9a5f5e1c None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:11.656099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04673666-6532-4465-8b91-25ad9a5f5e1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:11.657369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04673666-6532-4465-8b91-25ad9a5f5e1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.659860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04673666-6532-4465-8b91-25ad9a5f5e1c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:11.668495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950e33f-bbff-4203-a9e4-7db0dbe4fef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:11.669232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950e33f-bbff-4203-a9e4-7db0dbe4fef6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:11.669586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950e33f-bbff-4203-a9e4-7db0dbe4fef6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:11.669937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950e33f-bbff-4203-a9e4-7db0dbe4fef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:11.670883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950e33f-bbff-4203-a9e4-7db0dbe4fef6 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=4327e4eed5a1436ea1a00c2209bc5750) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:11.671264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950e33f-bbff-4203-a9e4-7db0dbe4fef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.672774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950e33f-bbff-4203-a9e4-7db0dbe4fef6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:11.678622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 271/535] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:11 2022] GET /identity/v3/role_assignments?user.id=4327e4eed5a1436ea1a00c2209bc5750&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:11.680050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950e33f-bbff-4203-a9e4-7db0dbe4fef6 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:11.686877 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 265/536] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:11.698219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04673666-6532-4465-8b91-25ad9a5f5e1c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:11.791140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:11.875940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950e33f-bbff-4203-a9e4-7db0dbe4fef6 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:11.881384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04673666-6532-4465-8b91-25ad9a5f5e1c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:11.919479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29392764-554c-4563-af28-0fbdd8fa4de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:11.920148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29392764-554c-4563-af28-0fbdd8fa4de5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:11.920601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29392764-554c-4563-af28-0fbdd8fa4de5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:11.920890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29392764-554c-4563-af28-0fbdd8fa4de5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:11.921743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29392764-554c-4563-af28-0fbdd8fa4de5 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=1bd66c1b7c7147a281ef2e7253224069) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:11.922057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29392764-554c-4563-af28-0fbdd8fa4de5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.923525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29392764-554c-4563-af28-0fbdd8fa4de5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:11.930422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 266/537] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:11 2022] GET /identity/v3/role_assignments?user.id=1bd66c1b7c7147a281ef2e7253224069&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:11.990035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4661f-5c95-4263-baa0-7881c8026cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:11.991687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4661f-5c95-4263-baa0-7881c8026cc8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:11.992493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4661f-5c95-4263-baa0-7881c8026cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:11.993274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4661f-5c95-4263-baa0-7881c8026cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:11.995016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4661f-5c95-4263-baa0-7881c8026cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:11.995872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4661f-5c95-4263-baa0-7881c8026cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:11.998679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4661f-5c95-4263-baa0-7881c8026cc8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:12.033082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 272/538] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:12.038134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29392764-554c-4563-af28-0fbdd8fa4de5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:12.163010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29392764-554c-4563-af28-0fbdd8fa4de5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:12.165468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:12.211360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fc1bf-17d9-46a9-a198-cdfc265c6fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:12.212157 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc1bf-17d9-46a9-a198-cdfc265c6fa1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:12.212440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc1bf-17d9-46a9-a198-cdfc265c6fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:12.212721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc1bf-17d9-46a9-a198-cdfc265c6fa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:12.221150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fc1bf-17d9-46a9-a198-cdfc265c6fa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:12.222583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fc1bf-17d9-46a9-a198-cdfc265c6fa1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-337fc1bf-17d9-46a9-a198-cdfc265c6fa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:12.237461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:12.239006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:12.240607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:12.240607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:12.240607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:12.240607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:12.240607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:12.240607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:12.240607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:12.240607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:12.240607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 267/539] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:12.245789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4661f-5c95-4263-baa0-7881c8026cc8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:12.305401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:12.365991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4661f-5c95-4263-baa0-7881c8026cc8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:12.401598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3c158-18ee-4d5c-a8ae-6b23e6dd5e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:12.402260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3c158-18ee-4d5c-a8ae-6b23e6dd5e65 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:12.402533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3c158-18ee-4d5c-a8ae-6b23e6dd5e65 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:12.402843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3c158-18ee-4d5c-a8ae-6b23e6dd5e65 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:12.403638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3c158-18ee-4d5c-a8ae-6b23e6dd5e65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:12.403942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3c158-18ee-4d5c-a8ae-6b23e6dd5e65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:12.405393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3c158-18ee-4d5c-a8ae-6b23e6dd5e65 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:12.418283 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 273/540] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:12.423553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fc1bf-17d9-46a9-a198-cdfc265c6fa1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:12.549816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fc1bf-17d9-46a9-a198-cdfc265c6fa1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:12.585145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c35ec3-4288-4ec3-9a3e-d88b85570ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:12.585829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c35ec3-4288-4ec3-9a3e-d88b85570ab3 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:12.586109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c35ec3-4288-4ec3-9a3e-d88b85570ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:12.586387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c35ec3-4288-4ec3-9a3e-d88b85570ab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:12.587254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c35ec3-4288-4ec3-9a3e-d88b85570ab3 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=bf1915b9919442d692bcedf25f53320c) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:12.587563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c35ec3-4288-4ec3-9a3e-d88b85570ab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:12.589052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c35ec3-4288-4ec3-9a3e-d88b85570ab3 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:12.595119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 268/541] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:12 2022] GET /identity/v3/role_assignments?user.id=bf1915b9919442d692bcedf25f53320c&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:12.867735 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad10f2e-7a16-467c-bf79-8e1e3c05739e None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:12.867953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad10f2e-7a16-467c-bf79-8e1e3c05739e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:12.868321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad10f2e-7a16-467c-bf79-8e1e3c05739e None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:12.869762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 274/542] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:12.875100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bf762-8491-4ba1-9812-3bac2156dfa5 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:12.875392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bf762-8491-4ba1-9812-3bac2156dfa5 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:12.875674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bf762-8491-4ba1-9812-3bac2156dfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:12.898852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-6c7bf762-8491-4ba1-9812-3bac2156dfa5 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:12.900038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6c7bf762-8491-4ba1-9812-3bac2156dfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:12.940618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:13.045340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 269/543] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:13.080468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad10f2e-7a16-467c-bf79-8e1e3c05739e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:13.204420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad10f2e-7a16-467c-bf79-8e1e3c05739e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:13.206198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:13.254255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f94526-41d4-4f13-94d4-cb5cfb923a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:13.255221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94526-41d4-4f13-94d4-cb5cfb923a7c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:13.255352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94526-41d4-4f13-94d4-cb5cfb923a7c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:13.255694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94526-41d4-4f13-94d4-cb5cfb923a7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:13.267679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f94526-41d4-4f13-94d4-cb5cfb923a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-b1f94526-41d4-4f13-94d4-cb5cfb923a7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:13.276135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:13.277723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:13.279311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:13.283006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 275/544] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:13.284221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bf762-8491-4ba1-9812-3bac2156dfa5 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:13.399381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bf762-8491-4ba1-9812-3bac2156dfa5 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:13.460710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8bf44-2b85-4274-96c6-28eb52d61d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:13.461409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8bf44-2b85-4274-96c6-28eb52d61d43 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:13.461692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8bf44-2b85-4274-96c6-28eb52d61d43 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:13.461966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8bf44-2b85-4274-96c6-28eb52d61d43 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:13.462751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8bf44-2b85-4274-96c6-28eb52d61d43 None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:13.463062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8bf44-2b85-4274-96c6-28eb52d61d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:13.464429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8bf44-2b85-4274-96c6-28eb52d61d43 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:13.485457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 270/545] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:13.491883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f94526-41d4-4f13-94d4-cb5cfb923a7c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:13.687144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f94526-41d4-4f13-94d4-cb5cfb923a7c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:13.745057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c1f0-2094-44a5-9960-8a890e38ad56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:13.746269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c1f0-2094-44a5-9960-8a890e38ad56 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:13.746808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c1f0-2094-44a5-9960-8a890e38ad56 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:13.747313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c1f0-2094-44a5-9960-8a890e38ad56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:13.766231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c1f0-2094-44a5-9960-8a890e38ad56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:13.769138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c1f0-2094-44a5-9960-8a890e38ad56 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-f557c1f0-2094-44a5-9960-8a890e38ad56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:13.801483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:13.803149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:13.805040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:13.805040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:13.805040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:13.805040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:13.805040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 276/546] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:13.809990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8bf44-2b85-4274-96c6-28eb52d61d43 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:13.850269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:13.952266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8bf44-2b85-4274-96c6-28eb52d61d43 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:13.994514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f164eb8-7409-44cd-80cd-a03e18ddccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:13.995268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f164eb8-7409-44cd-80cd-a03e18ddccda None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:13.995542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f164eb8-7409-44cd-80cd-a03e18ddccda None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:13.995786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f164eb8-7409-44cd-80cd-a03e18ddccda None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:13.996458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f164eb8-7409-44cd-80cd-a03e18ddccda None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:13.996762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f164eb8-7409-44cd-80cd-a03e18ddccda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:13.998104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f164eb8-7409-44cd-80cd-a03e18ddccda None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:14.013929 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 271/547] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:14.020503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c1f0-2094-44a5-9960-8a890e38ad56 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:14.063383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:14.195926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c1f0-2094-44a5-9960-8a890e38ad56 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:14.237141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6342d00-1b05-4af9-a6be-1c8ad8305b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:14.237945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6342d00-1b05-4af9-a6be-1c8ad8305b53 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:14.238231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6342d00-1b05-4af9-a6be-1c8ad8305b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:14.238508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6342d00-1b05-4af9-a6be-1c8ad8305b53 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:14.247329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6342d00-1b05-4af9-a6be-1c8ad8305b53 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:14.248729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6342d00-1b05-4af9-a6be-1c8ad8305b53 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-e6342d00-1b05-4af9-a6be-1c8ad8305b53 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:14.264786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:14.266176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:14.267620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 277/548] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:14.272162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f164eb8-7409-44cd-80cd-a03e18ddccda None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:14.373701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:14.401853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f164eb8-7409-44cd-80cd-a03e18ddccda None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:14.450156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038af7df-99f1-4c99-835d-423436b9df6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:14.451320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af7df-99f1-4c99-835d-423436b9df6e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:14.451320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af7df-99f1-4c99-835d-423436b9df6e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:14.451479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af7df-99f1-4c99-835d-423436b9df6e None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:14.454752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038af7df-99f1-4c99-835d-423436b9df6e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:14.454752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038af7df-99f1-4c99-835d-423436b9df6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:14.454752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038af7df-99f1-4c99-835d-423436b9df6e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:14.472706 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 272/549] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Jun 15 15:37:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:14.478236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6342d00-1b05-4af9-a6be-1c8ad8305b53 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:14.597533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:14.625375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6342d00-1b05-4af9-a6be-1c8ad8305b53 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:14.662856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5509cf6-8784-46a0-b7bb-25c4e8349c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:14.664120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5509cf6-8784-46a0-b7bb-25c4e8349c5d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:14.664464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5509cf6-8784-46a0-b7bb-25c4e8349c5d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:14.664746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5509cf6-8784-46a0-b7bb-25c4e8349c5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:14.736979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5509cf6-8784-46a0-b7bb-25c4e8349c5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:14.738542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5509cf6-8784-46a0-b7bb-25c4e8349c5d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:14.818437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:14.846181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 278/550] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:14.852995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038af7df-99f1-4c99-835d-423436b9df6e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:14.959307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca293-cad8-4bdd-a57c-329cd627942f None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:14.959591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca293-cad8-4bdd-a57c-329cd627942f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:14.960109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca293-cad8-4bdd-a57c-329cd627942f None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:14.961988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 279/551] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:14.967312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b5238-5932-4738-874f-4d7f6f071343 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:14.967973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b5238-5932-4738-874f-4d7f6f071343 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:14.968454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b5238-5932-4738-874f-4d7f6f071343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:14.993748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038af7df-99f1-4c99-835d-423436b9df6e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:15.005373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-333b5238-5932-4738-874f-4d7f6f071343 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:15.006929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-333b5238-5932-4738-874f-4d7f6f071343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:15.035675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d7a01-c4c7-42aa-bb0c-e3626567a9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:15.036738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d7a01-c4c7-42aa-bb0c-e3626567a9e2 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.037332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d7a01-c4c7-42aa-bb0c-e3626567a9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.039883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d7a01-c4c7-42aa-bb0c-e3626567a9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.044293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:15.061056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d7a01-c4c7-42aa-bb0c-e3626567a9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:15.062565 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d7a01-c4c7-42aa-bb0c-e3626567a9e2 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-2d9d7a01-c4c7-42aa-bb0c-e3626567a9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:15.089622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:15.090639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:15.091556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:15.091556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:15.091556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:15.091556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:15.091556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 273/552] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:15.094751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879705c-b3d3-476b-900b-90d1daf81a47 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.094751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879705c-b3d3-476b-900b-90d1daf81a47 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.094751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879705c-b3d3-476b-900b-90d1daf81a47 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.098771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 274/553] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:15.099689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879705c-b3d3-476b-900b-90d1daf81a47 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.125547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 280/554] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:15.127337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1693e7a-3a6d-4e07-bda0-7c1ac6ffd93a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.127812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1693e7a-3a6d-4e07-bda0-7c1ac6ffd93a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.128306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1693e7a-3a6d-4e07-bda0-7c1ac6ffd93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.148331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e1693e7a-3a6d-4e07-bda0-7c1ac6ffd93a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:15.150062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e1693e7a-3a6d-4e07-bda0-7c1ac6ffd93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:15.213920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879705c-b3d3-476b-900b-90d1daf81a47 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:15.225429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 281/555] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:15.228589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af43bb-ffbc-48fd-bca2-e49975e14b42 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.229062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af43bb-ffbc-48fd-bca2-e49975e14b42 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.229748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af43bb-ffbc-48fd-bca2-e49975e14b42 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.236388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 282/556] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:15.237836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af43bb-ffbc-48fd-bca2-e49975e14b42 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.261456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83ea95-96ba-4757-a740-ab4b1825d1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:15.262117 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83ea95-96ba-4757-a740-ab4b1825d1ef None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.262408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83ea95-96ba-4757-a740-ab4b1825d1ef None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.262687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83ea95-96ba-4757-a740-ab4b1825d1ef None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.263444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83ea95-96ba-4757-a740-ab4b1825d1ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:15.263753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83ea95-96ba-4757-a740-ab4b1825d1ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:15.265182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83ea95-96ba-4757-a740-ab4b1825d1ef None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:15.275412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:15.278234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 275/557] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:15.279163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5cc8f-4919-41ec-a7e7-e5ca05a1f17d None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.279501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5cc8f-4919-41ec-a7e7-e5ca05a1f17d None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.279737 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5cc8f-4919-41ec-a7e7-e5ca05a1f17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.297782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-dec5cc8f-4919-41ec-a7e7-e5ca05a1f17d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:15.298686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-dec5cc8f-4919-41ec-a7e7-e5ca05a1f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:15.344463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:15.354115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af43bb-ffbc-48fd-bca2-e49975e14b42 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:15.372829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 276/558] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:15.374052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5cc8f-4919-41ec-a7e7-e5ca05a1f17d None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.399698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c482f56-4204-4d5c-912d-a4dc21767c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:15.400424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c482f56-4204-4d5c-912d-a4dc21767c0e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.400703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c482f56-4204-4d5c-912d-a4dc21767c0e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.400985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c482f56-4204-4d5c-912d-a4dc21767c0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.411722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c482f56-4204-4d5c-912d-a4dc21767c0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-8c482f56-4204-4d5c-912d-a4dc21767c0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:15.419804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:15.420724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:15.422099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 283/559] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:15.422928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c482f56-4204-4d5c-912d-a4dc21767c0e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.494506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5cc8f-4919-41ec-a7e7-e5ca05a1f17d None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:15.518499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:15.526439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c482f56-4204-4d5c-912d-a4dc21767c0e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:15.528562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f22421-0d0c-47f8-8d91-972239d32e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:15.529347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22421-0d0c-47f8-8d91-972239d32e52 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.529676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22421-0d0c-47f8-8d91-972239d32e52 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.529955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22421-0d0c-47f8-8d91-972239d32e52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.539959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f22421-0d0c-47f8-8d91-972239d32e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-f8f22421-0d0c-47f8-8d91-972239d32e52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:15.547503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:15.548879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 277/560] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:15.550113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f22421-0d0c-47f8-8d91-972239d32e52 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.564748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963ce3d-f222-4e90-8e7a-49be2741c8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:15.565550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963ce3d-f222-4e90-8e7a-49be2741c8ca None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.565838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963ce3d-f222-4e90-8e7a-49be2741c8ca None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.566112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963ce3d-f222-4e90-8e7a-49be2741c8ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.573459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963ce3d-f222-4e90-8e7a-49be2741c8ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:15.574954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963ce3d-f222-4e90-8e7a-49be2741c8ca None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:15.587664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-d963ce3d-f222-4e90-8e7a-49be2741c8ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:15.589012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:15.590181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:15.591429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:15.591429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:15.591429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:15.591429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:15.591429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:15.591429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:15.591429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:15.591429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:15.591429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 284/561] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:15.592026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594402d-afd2-44b3-9e79-affc27db0ce8 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.592026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594402d-afd2-44b3-9e79-affc27db0ce8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.592236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594402d-afd2-44b3-9e79-affc27db0ce8 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.593495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 285/562] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:15.594184 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594402d-afd2-44b3-9e79-affc27db0ce8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.651772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f22421-0d0c-47f8-8d91-972239d32e52 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:15.686375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd830a-0592-4d46-8b84-520be06da975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:15.687307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd830a-0592-4d46-8b84-520be06da975 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.687603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd830a-0592-4d46-8b84-520be06da975 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.687885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd830a-0592-4d46-8b84-520be06da975 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.693889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594402d-afd2-44b3-9e79-affc27db0ce8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:15.698696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd830a-0592-4d46-8b84-520be06da975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-1edd830a-0592-4d46-8b84-520be06da975 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:15.706402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:15.707594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:15.708536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 278/563] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:15.709225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd830a-0592-4d46-8b84-520be06da975 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.731694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75044bc6-dabe-4495-8172-4b8c576da213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:15.732356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75044bc6-dabe-4495-8172-4b8c576da213 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.732641 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75044bc6-dabe-4495-8172-4b8c576da213 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.732921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75044bc6-dabe-4495-8172-4b8c576da213 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.733663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75044bc6-dabe-4495-8172-4b8c576da213 None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:15.733974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75044bc6-dabe-4495-8172-4b8c576da213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:15.735394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75044bc6-dabe-4495-8172-4b8c576da213 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:15.743395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 286/564] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:15.744170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75044bc6-dabe-4495-8172-4b8c576da213 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.763837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:15.805436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:15.808940 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd830a-0592-4d46-8b84-520be06da975 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:15.842233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9652a0a-60d9-4c24-bede-1bc3469a4976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:15.842914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9652a0a-60d9-4c24-bede-1bc3469a4976 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.843198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9652a0a-60d9-4c24-bede-1bc3469a4976 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.843472 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9652a0a-60d9-4c24-bede-1bc3469a4976 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.844215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9652a0a-60d9-4c24-bede-1bc3469a4976 None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:15.844520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9652a0a-60d9-4c24-bede-1bc3469a4976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:15.844670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75044bc6-dabe-4495-8172-4b8c576da213 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:15.845868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9652a0a-60d9-4c24-bede-1bc3469a4976 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:15.854032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 279/565] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:15.855434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb46671-9e00-4742-8abc-d34dda67e4e2 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.855723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb46671-9e00-4742-8abc-d34dda67e4e2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.856000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb46671-9e00-4742-8abc-d34dda67e4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.872660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-5cb46671-9e00-4742-8abc-d34dda67e4e2 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:15.873822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5cb46671-9e00-4742-8abc-d34dda67e4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:15.879083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628e525-31c6-4996-857e-db441e19ffc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:15.879741 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628e525-31c6-4996-857e-db441e19ffc7 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:15.880024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628e525-31c6-4996-857e-db441e19ffc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:15.880298 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628e525-31c6-4996-857e-db441e19ffc7 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:15.881478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628e525-31c6-4996-857e-db441e19ffc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:15.881790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628e525-31c6-4996-857e-db441e19ffc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:15.884146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628e525-31c6-4996-857e-db441e19ffc7 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:15.901394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 287/566] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:15.901394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628e525-31c6-4996-857e-db441e19ffc7 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.945041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 280/567] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:15.945885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb46671-9e00-4742-8abc-d34dda67e4e2 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:15.980165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:16.004180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628e525-31c6-4996-857e-db441e19ffc7 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.037098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:16.039273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f20da-18fd-42c0-84a5-982a6486f2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.040067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f20da-18fd-42c0-84a5-982a6486f2ee None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.040348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f20da-18fd-42c0-84a5-982a6486f2ee None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.040615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f20da-18fd-42c0-84a5-982a6486f2ee None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.041347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f20da-18fd-42c0-84a5-982a6486f2ee None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:16.041647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f20da-18fd-42c0-84a5-982a6486f2ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.043037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f20da-18fd-42c0-84a5-982a6486f2ee None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.047328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb46671-9e00-4742-8abc-d34dda67e4e2 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.050719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 288/568] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:16.051503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f20da-18fd-42c0-84a5-982a6486f2ee None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.079989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccce03-e06e-4ceb-8b07-fac9cf475e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.080769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccce03-e06e-4ceb-8b07-fac9cf475e99 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.081051 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccce03-e06e-4ceb-8b07-fac9cf475e99 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.081326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccce03-e06e-4ceb-8b07-fac9cf475e99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.091598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccce03-e06e-4ceb-8b07-fac9cf475e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.093036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccce03-e06e-4ceb-8b07-fac9cf475e99 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-b3ccce03-e06e-4ceb-8b07-fac9cf475e99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:16.110396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:16.111466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:16.112946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:16.112946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:16.112946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:16.112946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:16.112946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 281/569] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:16.112946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccce03-e06e-4ceb-8b07-fac9cf475e99 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.154665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f20da-18fd-42c0-84a5-982a6486f2ee None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.187919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd95d7-d9f3-45be-9b11-3f994645e3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.188703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd95d7-d9f3-45be-9b11-3f994645e3ed None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.188989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd95d7-d9f3-45be-9b11-3f994645e3ed None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.189266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd95d7-d9f3-45be-9b11-3f994645e3ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.195837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:16.201013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd95d7-d9f3-45be-9b11-3f994645e3ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.202532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd95d7-d9f3-45be-9b11-3f994645e3ed None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.212141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccce03-e06e-4ceb-8b07-fac9cf475e99 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-aebd95d7-d9f3-45be-9b11-3f994645e3ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:16.219783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:16.220655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:16.226341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:16.226341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:16.226341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:16.226341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:16.226341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 289/570] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:16.226341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd95d7-d9f3-45be-9b11-3f994645e3ed None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.246134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282cd9e-eb74-45b9-827b-41786ed305fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.246821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cd9e-eb74-45b9-827b-41786ed305fc None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.247106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cd9e-eb74-45b9-827b-41786ed305fc None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.247382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cd9e-eb74-45b9-827b-41786ed305fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.248219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282cd9e-eb74-45b9-827b-41786ed305fc None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=a6e3bb6553264387936e9a534afc3ac0) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:16.248533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282cd9e-eb74-45b9-827b-41786ed305fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.249951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282cd9e-eb74-45b9-827b-41786ed305fc None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.255454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 282/571] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/role_assignments?user.id=a6e3bb6553264387936e9a534afc3ac0&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:16.256223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282cd9e-eb74-45b9-827b-41786ed305fc None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.265648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:16.351846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd95d7-d9f3-45be-9b11-3f994645e3ed None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.384406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282cd9e-eb74-45b9-827b-41786ed305fc None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.390224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1193ecc-43da-4861-b2ff-47abfb2776a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.391166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1193ecc-43da-4861-b2ff-47abfb2776a5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.391521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1193ecc-43da-4861-b2ff-47abfb2776a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.392409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1193ecc-43da-4861-b2ff-47abfb2776a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.406518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1193ecc-43da-4861-b2ff-47abfb2776a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-a1193ecc-43da-4861-b2ff-47abfb2776a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:16.415595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:16.416716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 290/572] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:16.418387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1193ecc-43da-4861-b2ff-47abfb2776a5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.430563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6add-e361-4e8e-9cc4-52c22ee5a908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.432050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6add-e361-4e8e-9cc4-52c22ee5a908 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.432480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6add-e361-4e8e-9cc4-52c22ee5a908 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.432917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6add-e361-4e8e-9cc4-52c22ee5a908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.447826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa6add-e361-4e8e-9cc4-52c22ee5a908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.449392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa6add-e361-4e8e-9cc4-52c22ee5a908 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-86aa6add-e361-4e8e-9cc4-52c22ee5a908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:16.467173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:16.468181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:16.474352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:16.474352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:16.474352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:16.474352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:16.474352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 283/573] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:16.474352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6add-e361-4e8e-9cc4-52c22ee5a908 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.485791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:16.531622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:16.540609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1193ecc-43da-4861-b2ff-47abfb2776a5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.575653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6add-e361-4e8e-9cc4-52c22ee5a908 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.580911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaf0b5-1590-4504-ac31-59a7d4d2f6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.581590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf0b5-1590-4504-ac31-59a7d4d2f6bc None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.581875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf0b5-1590-4504-ac31-59a7d4d2f6bc None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.582150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf0b5-1590-4504-ac31-59a7d4d2f6bc None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.582904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaf0b5-1590-4504-ac31-59a7d4d2f6bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:16.583217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaf0b5-1590-4504-ac31-59a7d4d2f6bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.584591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaf0b5-1590-4504-ac31-59a7d4d2f6bc None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.597936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 291/574] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:16.598772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaf0b5-1590-4504-ac31-59a7d4d2f6bc None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.616916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac6b29-9385-4190-b1b1-b0212eb34249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.617799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac6b29-9385-4190-b1b1-b0212eb34249 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.618231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac6b29-9385-4190-b1b1-b0212eb34249 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.618652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac6b29-9385-4190-b1b1-b0212eb34249 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.619563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac6b29-9385-4190-b1b1-b0212eb34249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:16.620072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac6b29-9385-4190-b1b1-b0212eb34249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.621640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac6b29-9385-4190-b1b1-b0212eb34249 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.634784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 284/575] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:16.635884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac6b29-9385-4190-b1b1-b0212eb34249 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.706311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaf0b5-1590-4504-ac31-59a7d4d2f6bc None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.718457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:16.742939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac6b29-9385-4190-b1b1-b0212eb34249 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.744008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452acd4-54e8-4886-b3ae-9a8fee5b7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.744679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452acd4-54e8-4886-b3ae-9a8fee5b7562 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.744960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452acd4-54e8-4886-b3ae-9a8fee5b7562 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.745245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452acd4-54e8-4886-b3ae-9a8fee5b7562 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.745982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452acd4-54e8-4886-b3ae-9a8fee5b7562 None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:16.746291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452acd4-54e8-4886-b3ae-9a8fee5b7562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.747702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452acd4-54e8-4886-b3ae-9a8fee5b7562 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.757746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 292/576] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:16.758386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452acd4-54e8-4886-b3ae-9a8fee5b7562 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.764595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:16.783271 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955cc58-ccc9-48aa-ba51-537c83b07003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.783938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955cc58-ccc9-48aa-ba51-537c83b07003 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.784207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955cc58-ccc9-48aa-ba51-537c83b07003 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.784487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955cc58-ccc9-48aa-ba51-537c83b07003 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.785201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955cc58-ccc9-48aa-ba51-537c83b07003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:16.785503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955cc58-ccc9-48aa-ba51-537c83b07003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.786888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955cc58-ccc9-48aa-ba51-537c83b07003 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.798803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 285/577] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:16.799956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955cc58-ccc9-48aa-ba51-537c83b07003 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.870955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452acd4-54e8-4886-b3ae-9a8fee5b7562 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.904716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdc5a0-418e-4f76-af5b-d0b5eae4fa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.905509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdc5a0-418e-4f76-af5b-d0b5eae4fa7f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.905818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdc5a0-418e-4f76-af5b-d0b5eae4fa7f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.906077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdc5a0-418e-4f76-af5b-d0b5eae4fa7f None admin] PATH_INFO: `/v3/users/glance` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.908376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955cc58-ccc9-48aa-ba51-537c83b07003 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:16.913745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbdc5a0-418e-4f76-af5b-d0b5eae4fa7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.915241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbdc5a0-418e-4f76-af5b-d0b5eae4fa7f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-7cbdc5a0-418e-4f76-af5b-d0b5eae4fa7f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:16.929689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:16.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 15:37:16.931834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:16.935517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 293/578] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:16.936362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdc5a0-418e-4f76-af5b-d0b5eae4fa7f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:16.948279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e6b6-1d01-453d-82d8-40c9828abf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:16.949464 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e6b6-1d01-453d-82d8-40c9828abf51 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:16.949909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e6b6-1d01-453d-82d8-40c9828abf51 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:16.950193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e6b6-1d01-453d-82d8-40c9828abf51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:16.958290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09e6b6-1d01-453d-82d8-40c9828abf51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:16.959877 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09e6b6-1d01-453d-82d8-40c9828abf51 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:16.967688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-7d09e6b6-1d01-453d-82d8-40c9828abf51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:16.976037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:16.976955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:16.977884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 286/579] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:16.978464 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e6b6-1d01-453d-82d8-40c9828abf51 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.020869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:17.044617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdc5a0-418e-4f76-af5b-d0b5eae4fa7f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.080274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a365f1a-b00f-4244-909f-37b04b2692e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.081607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a365f1a-b00f-4244-909f-37b04b2692e3 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.081907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a365f1a-b00f-4244-909f-37b04b2692e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.082182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a365f1a-b00f-4244-909f-37b04b2692e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.089522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e6b6-1d01-453d-82d8-40c9828abf51 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.095417 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a365f1a-b00f-4244-909f-37b04b2692e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.097248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a365f1a-b00f-4244-909f-37b04b2692e3 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-4a365f1a-b00f-4244-909f-37b04b2692e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:17.116838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:17.117740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:17.118626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:17.118626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:17.118626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:17.118626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:17.118626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 294/580] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:17.119395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a365f1a-b00f-4244-909f-37b04b2692e3 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.130333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad9377-b7ee-4da0-b2bb-55e62823c6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.131162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9377-b7ee-4da0-b2bb-55e62823c6fa None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.131473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9377-b7ee-4da0-b2bb-55e62823c6fa None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.131727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9377-b7ee-4da0-b2bb-55e62823c6fa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.139006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad9377-b7ee-4da0-b2bb-55e62823c6fa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.140436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad9377-b7ee-4da0-b2bb-55e62823c6fa None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-bbad9377-b7ee-4da0-b2bb-55e62823c6fa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:17.155948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:17.157160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 15:37:17.158335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:17.161592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 287/581] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:17.162395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad9377-b7ee-4da0-b2bb-55e62823c6fa None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.213102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:17.231057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a365f1a-b00f-4244-909f-37b04b2692e3 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.267452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d665040-c0e1-4502-a60c-8e852eef4ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.268217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d665040-c0e1-4502-a60c-8e852eef4ccb None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.268409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d665040-c0e1-4502-a60c-8e852eef4ccb None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.268688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d665040-c0e1-4502-a60c-8e852eef4ccb None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.269000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:17.269525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d665040-c0e1-4502-a60c-8e852eef4ccb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:17.269974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d665040-c0e1-4502-a60c-8e852eef4ccb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.272070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d665040-c0e1-4502-a60c-8e852eef4ccb None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.273880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad9377-b7ee-4da0-b2bb-55e62823c6fa None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.290813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 295/582] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:17.291464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d665040-c0e1-4502-a60c-8e852eef4ccb None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.315015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e330354-7f88-4517-a3eb-01c1e7e651ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.315710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e330354-7f88-4517-a3eb-01c1e7e651ed None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.316000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e330354-7f88-4517-a3eb-01c1e7e651ed None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.316412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e330354-7f88-4517-a3eb-01c1e7e651ed None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.317146 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e330354-7f88-4517-a3eb-01c1e7e651ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:17.317473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e330354-7f88-4517-a3eb-01c1e7e651ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.318992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e330354-7f88-4517-a3eb-01c1e7e651ed None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.339326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 288/583] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:17.340174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e330354-7f88-4517-a3eb-01c1e7e651ed None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.405606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d665040-c0e1-4502-a60c-8e852eef4ccb None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.449053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b045c09-29db-4140-86e1-4e88e71902e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.449846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e330354-7f88-4517-a3eb-01c1e7e651ed None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.450748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b045c09-29db-4140-86e1-4e88e71902e2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.451318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b045c09-29db-4140-86e1-4e88e71902e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.451800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b045c09-29db-4140-86e1-4e88e71902e2 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.452800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b045c09-29db-4140-86e1-4e88e71902e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:17.453191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b045c09-29db-4140-86e1-4e88e71902e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.454595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b045c09-29db-4140-86e1-4e88e71902e2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.471532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 296/584] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:17.471532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b045c09-29db-4140-86e1-4e88e71902e2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.474339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:17.491119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e754dac-194d-42f9-bade-a6c7b0c55557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.492476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e754dac-194d-42f9-bade-a6c7b0c55557 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.493210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e754dac-194d-42f9-bade-a6c7b0c55557 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.496004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e754dac-194d-42f9-bade-a6c7b0c55557 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.496871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e754dac-194d-42f9-bade-a6c7b0c55557 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:17.497511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e754dac-194d-42f9-bade-a6c7b0c55557 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.498709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e754dac-194d-42f9-bade-a6c7b0c55557 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.516945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 289/585] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:17.517573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e754dac-194d-42f9-bade-a6c7b0c55557 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.533145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:17.589373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b045c09-29db-4140-86e1-4e88e71902e2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.622555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e754dac-194d-42f9-bade-a6c7b0c55557 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.630255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50c26f-64a8-4d71-bc95-2ebe5156943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.630974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50c26f-64a8-4d71-bc95-2ebe5156943d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.631262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50c26f-64a8-4d71-bc95-2ebe5156943d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.631548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50c26f-64a8-4d71-bc95-2ebe5156943d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.642403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50c26f-64a8-4d71-bc95-2ebe5156943d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.643894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50c26f-64a8-4d71-bc95-2ebe5156943d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.657875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c657ed-04c3-44bd-97e4-d31dcda64512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.658647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c657ed-04c3-44bd-97e4-d31dcda64512 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.658957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c657ed-04c3-44bd-97e4-d31dcda64512 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.659231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c657ed-04c3-44bd-97e4-d31dcda64512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-fd50c26f-64a8-4d71-bc95-2ebe5156943d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:17.661930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:17.662842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:17.664290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:17.664290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:17.664290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:17.664290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:17.664290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 297/586] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:17.664290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50c26f-64a8-4d71-bc95-2ebe5156943d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.675849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c657ed-04c3-44bd-97e4-d31dcda64512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.676973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c657ed-04c3-44bd-97e4-d31dcda64512 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-56c657ed-04c3-44bd-97e4-d31dcda64512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:17.695081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:17.696163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:17.697218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:17.697218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:17.697218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:17.697218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:17.697218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 290/587] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:17.701217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c657ed-04c3-44bd-97e4-d31dcda64512 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.727482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:17.769281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:17.777121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50c26f-64a8-4d71-bc95-2ebe5156943d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.806234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c657ed-04c3-44bd-97e4-d31dcda64512 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.813814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3476f-d792-4994-95e0-4ba969e44cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.814686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3476f-d792-4994-95e0-4ba969e44cf4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.815076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3476f-d792-4994-95e0-4ba969e44cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.815531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3476f-d792-4994-95e0-4ba969e44cf4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.823251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3476f-d792-4994-95e0-4ba969e44cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.824772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3476f-d792-4994-95e0-4ba969e44cf4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-01a3476f-d792-4994-95e0-4ba969e44cf4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:17.839708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:17.840862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:17.841879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 298/588] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:17.842468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3476f-d792-4994-95e0-4ba969e44cf4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.843791 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae749e7-1e3d-42dd-9441-44f2187e667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:17.844579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae749e7-1e3d-42dd-9441-44f2187e667f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:17.844861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae749e7-1e3d-42dd-9441-44f2187e667f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:17.845139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae749e7-1e3d-42dd-9441-44f2187e667f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:17.857258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae749e7-1e3d-42dd-9441-44f2187e667f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:17.858705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae749e7-1e3d-42dd-9441-44f2187e667f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-eae749e7-1e3d-42dd-9441-44f2187e667f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:17.876679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:17.877771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:17.878687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:17.878687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:17.878687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:17.878687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:17.878687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 291/589] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:17.878687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae749e7-1e3d-42dd-9441-44f2187e667f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:17.967685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3476f-d792-4994-95e0-4ba969e44cf4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:17.983426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:17.984362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae749e7-1e3d-42dd-9441-44f2187e667f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.003618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:18.006919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6939f3-d505-4817-abcf-e8f96ee55118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.007608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6939f3-d505-4817-abcf-e8f96ee55118 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.007879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6939f3-d505-4817-abcf-e8f96ee55118 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.008164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6939f3-d505-4817-abcf-e8f96ee55118 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.008888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6939f3-d505-4817-abcf-e8f96ee55118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:18.009202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6939f3-d505-4817-abcf-e8f96ee55118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.010590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6939f3-d505-4817-abcf-e8f96ee55118 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.023457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b79bf-f872-4b6d-b27e-88f3f48d82a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.027866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 299/590] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:18.027972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b79bf-f872-4b6d-b27e-88f3f48d82a1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.027972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b79bf-f872-4b6d-b27e-88f3f48d82a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.028101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6939f3-d505-4817-abcf-e8f96ee55118 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.028178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b79bf-f872-4b6d-b27e-88f3f48d82a1 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.028178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b79bf-f872-4b6d-b27e-88f3f48d82a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:18.028178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b79bf-f872-4b6d-b27e-88f3f48d82a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.028178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b79bf-f872-4b6d-b27e-88f3f48d82a1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.044458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 292/591] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:18.044458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b79bf-f872-4b6d-b27e-88f3f48d82a1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.144200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b79bf-f872-4b6d-b27e-88f3f48d82a1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.149569 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6939f3-d505-4817-abcf-e8f96ee55118 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.184034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72dddf7-beb6-4513-8ef2-12f8b41e00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.184938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dddf7-beb6-4513-8ef2-12f8b41e00af None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.185248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dddf7-beb6-4513-8ef2-12f8b41e00af None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.185739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dddf7-beb6-4513-8ef2-12f8b41e00af None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.186491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dddf7-beb6-4513-8ef2-12f8b41e00af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:18.186822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dddf7-beb6-4513-8ef2-12f8b41e00af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.188436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dddf7-beb6-4513-8ef2-12f8b41e00af None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.189448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e56a86-6f2b-44eb-b7d3-cb65c5f6c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.190240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e56a86-6f2b-44eb-b7d3-cb65c5f6c3cf None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.190625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e56a86-6f2b-44eb-b7d3-cb65c5f6c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.191032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e56a86-6f2b-44eb-b7d3-cb65c5f6c3cf None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.192006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e56a86-6f2b-44eb-b7d3-cb65c5f6c3cf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:18.192415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e56a86-6f2b-44eb-b7d3-cb65c5f6c3cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.193942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e56a86-6f2b-44eb-b7d3-cb65c5f6c3cf None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.205030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 293/592] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:18.205886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72dddf7-beb6-4513-8ef2-12f8b41e00af None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.216437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 300/593] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:18.216437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e56a86-6f2b-44eb-b7d3-cb65c5f6c3cf None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.247679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:18.252197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:18.319607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72dddf7-beb6-4513-8ef2-12f8b41e00af None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.325747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e56a86-6f2b-44eb-b7d3-cb65c5f6c3cf None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.356307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb568213-84b4-4525-be64-7db2e703a121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.357093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb568213-84b4-4525-be64-7db2e703a121 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.357371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb568213-84b4-4525-be64-7db2e703a121 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.357649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb568213-84b4-4525-be64-7db2e703a121 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.362545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4873-0af2-46ec-b4aa-914f3049d195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.363493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4873-0af2-46ec-b4aa-914f3049d195 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.363880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4873-0af2-46ec-b4aa-914f3049d195 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.364257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4873-0af2-46ec-b4aa-914f3049d195 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.364399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb568213-84b4-4525-be64-7db2e703a121 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.365830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb568213-84b4-4525-be64-7db2e703a121 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.376819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d4873-0af2-46ec-b4aa-914f3049d195 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.384124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d4873-0af2-46ec-b4aa-914f3049d195 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-bb568213-84b4-4525-be64-7db2e703a121 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:18.384564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:18.385583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:18.386627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:18.386627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:18.386627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:18.386627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:18.386627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:18.386627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:18.386627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:18.386627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:18.386627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 294/594] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:18.387371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb568213-84b4-4525-be64-7db2e703a121 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-844d4873-0af2-46ec-b4aa-914f3049d195 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:18.401155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:18.402251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 301/595] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:18.403364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4873-0af2-46ec-b4aa-914f3049d195 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.490491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:18.495379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb568213-84b4-4525-be64-7db2e703a121 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.495992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:18.508787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4873-0af2-46ec-b4aa-914f3049d195 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.531842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9918c4-ddc2-45ca-9c62-d169dc4ab12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.532631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9918c4-ddc2-45ca-9c62-d169dc4ab12b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.532927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9918c4-ddc2-45ca-9c62-d169dc4ab12b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.533205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9918c4-ddc2-45ca-9c62-d169dc4ab12b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.539920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9918c4-ddc2-45ca-9c62-d169dc4ab12b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.541466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9918c4-ddc2-45ca-9c62-d169dc4ab12b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.543267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e68978-2383-48bd-85c1-fcb9f4d52ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.544054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e68978-2383-48bd-85c1-fcb9f4d52ed1 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.544395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e68978-2383-48bd-85c1-fcb9f4d52ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.544673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e68978-2383-48bd-85c1-fcb9f4d52ed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.555522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e68978-2383-48bd-85c1-fcb9f4d52ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-0a9918c4-ddc2-45ca-9c62-d169dc4ab12b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:18.556004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:18.557021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:18.557974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:18.557974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:18.557974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:18.557974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:18.557974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:18.557974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:18.557974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:18.557974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:18.558422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e68978-2383-48bd-85c1-fcb9f4d52ed1 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.558494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 295/596] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:18.558684 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b571-2c22-453a-bc31-539cc2237944 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.558998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b571-2c22-453a-bc31-539cc2237944 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.559278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b571-2c22-453a-bc31-539cc2237944 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.560537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 296/597] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:18.561221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9b571-2c22-453a-bc31-539cc2237944 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-b6e68978-2383-48bd-85c1-fcb9f4d52ed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:18.578535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:18.579674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:18.580766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:18.580766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:18.580766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:18.580766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:18.580766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 302/598] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:18.580766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e68978-2383-48bd-85c1-fcb9f4d52ed1 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.664189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9b571-2c22-453a-bc31-539cc2237944 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.683214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e68978-2383-48bd-85c1-fcb9f4d52ed1 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.698278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39e903-b944-44b2-b61f-522539c33f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.698957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39e903-b944-44b2-b61f-522539c33f07 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.699242 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39e903-b944-44b2-b61f-522539c33f07 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.699520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39e903-b944-44b2-b61f-522539c33f07 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.700308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39e903-b944-44b2-b61f-522539c33f07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:18.700618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39e903-b944-44b2-b61f-522539c33f07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.702035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39e903-b944-44b2-b61f-522539c33f07 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.713952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 297/599] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:18.714851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39e903-b944-44b2-b61f-522539c33f07 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.720314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1becd-eff4-441c-b8f3-a9399f5e55aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.720975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1becd-eff4-441c-b8f3-a9399f5e55aa None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.721263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1becd-eff4-441c-b8f3-a9399f5e55aa None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.721539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1becd-eff4-441c-b8f3-a9399f5e55aa None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.722324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1becd-eff4-441c-b8f3-a9399f5e55aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:18.722645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1becd-eff4-441c-b8f3-a9399f5e55aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.724110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1becd-eff4-441c-b8f3-a9399f5e55aa None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.733169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:18.736594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 303/600] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:18.737729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1bc20-b99f-4943-b157-ebc9f361914c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.738014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1bc20-b99f-4943-b157-ebc9f361914c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.738294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1bc20-b99f-4943-b157-ebc9f361914c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.743928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:18.754760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-42d1bc20-b99f-4943-b157-ebc9f361914c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:18.756119 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-42d1bc20-b99f-4943-b157-ebc9f361914c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:18.813330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39e903-b944-44b2-b61f-522539c33f07 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.822853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 304/601] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:18.823687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1bc20-b99f-4943-b157-ebc9f361914c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.861566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ba3a1-37d9-4069-b5ef-ea1dc11627f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.862324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ba3a1-37d9-4069-b5ef-ea1dc11627f8 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.862609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ba3a1-37d9-4069-b5ef-ea1dc11627f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.862922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ba3a1-37d9-4069-b5ef-ea1dc11627f8 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.863712 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ba3a1-37d9-4069-b5ef-ea1dc11627f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:18.864026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ba3a1-37d9-4069-b5ef-ea1dc11627f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.865461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ba3a1-37d9-4069-b5ef-ea1dc11627f8 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.878513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 298/602] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:18.879415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ba3a1-37d9-4069-b5ef-ea1dc11627f8 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.927225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1bc20-b99f-4943-b157-ebc9f361914c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:18.957231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:18.960038 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702adf99-ef5e-4e31-b7d7-776f1be68d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:18.960693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702adf99-ef5e-4e31-b7d7-776f1be68d28 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:18.960976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702adf99-ef5e-4e31-b7d7-776f1be68d28 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:18.961255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702adf99-ef5e-4e31-b7d7-776f1be68d28 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:18.961968 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702adf99-ef5e-4e31-b7d7-776f1be68d28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:18.962273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702adf99-ef5e-4e31-b7d7-776f1be68d28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:18.963656 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702adf99-ef5e-4e31-b7d7-776f1be68d28 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:18.965091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:18.976760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 305/603] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:18.977539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702adf99-ef5e-4e31-b7d7-776f1be68d28 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:18.984211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ba3a1-37d9-4069-b5ef-ea1dc11627f8 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:19.017321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525b2a0-5382-46cc-87ac-876e6f4ac95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:19.018188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525b2a0-5382-46cc-87ac-876e6f4ac95c None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:19.018469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525b2a0-5382-46cc-87ac-876e6f4ac95c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:19.018780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525b2a0-5382-46cc-87ac-876e6f4ac95c None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/5f7405c8bc8e4c84bb056cd9e1df61c7/roles/01737d3768e2407780ff51b345e1d020` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:19.032912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525b2a0-5382-46cc-87ac-876e6f4ac95c None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=5f7405c8bc8e4c84bb056cd9e1df61c7, role_id=01737d3768e2407780ff51b345e1d020)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:19.034327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525b2a0-5382-46cc-87ac-876e6f4ac95c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:19.046852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 299/604] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:37:18 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/5f7405c8bc8e4c84bb056cd9e1df61c7/roles/01737d3768e2407780ff51b345e1d020 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:37:19.047710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525b2a0-5382-46cc-87ac-876e6f4ac95c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:19.079311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702adf99-ef5e-4e31-b7d7-776f1be68d28 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:19.115681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803647-ec5a-4e42-b54d-2197ba438687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:19.117165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803647-ec5a-4e42-b54d-2197ba438687 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:19.117479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803647-ec5a-4e42-b54d-2197ba438687 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:19.117755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803647-ec5a-4e42-b54d-2197ba438687 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/4327e4eed5a1436ea1a00c2209bc5750/roles/01737d3768e2407780ff51b345e1d020` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:19.133569 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d803647-ec5a-4e42-b54d-2197ba438687 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=4327e4eed5a1436ea1a00c2209bc5750, role_id=01737d3768e2407780ff51b345e1d020)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:19.135093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d803647-ec5a-4e42-b54d-2197ba438687 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:19.152066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 306/605] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:37:18 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/4327e4eed5a1436ea1a00c2209bc5750/roles/01737d3768e2407780ff51b345e1d020 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:37:19.154005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803647-ec5a-4e42-b54d-2197ba438687 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:19.178188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525b2a0-5382-46cc-87ac-876e6f4ac95c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:19.223542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:19.235546 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321d96f-1f71-46cb-9680-866634fd5ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:19.237314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321d96f-1f71-46cb-9680-866634fd5ff3 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:19.237818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321d96f-1f71-46cb-9680-866634fd5ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:19.238321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321d96f-1f71-46cb-9680-866634fd5ff3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:19.248047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:19.255109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321d96f-1f71-46cb-9680-866634fd5ff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-e321d96f-1f71-46cb-9680-866634fd5ff3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:19.263300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:19.264288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:19.265111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 300/606] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:19.265797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321d96f-1f71-46cb-9680-866634fd5ff3 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:19.321401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803647-ec5a-4e42-b54d-2197ba438687 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:19.362526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c12de-2bce-4ce1-89af-f689adcd7862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:19.363391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c12de-2bce-4ce1-89af-f689adcd7862 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:19.363675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c12de-2bce-4ce1-89af-f689adcd7862 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:19.363958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c12de-2bce-4ce1-89af-f689adcd7862 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/1bd66c1b7c7147a281ef2e7253224069/roles/01737d3768e2407780ff51b345e1d020` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:19.382188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c12de-2bce-4ce1-89af-f689adcd7862 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=1bd66c1b7c7147a281ef2e7253224069, role_id=01737d3768e2407780ff51b345e1d020)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:19.383886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c12de-2bce-4ce1-89af-f689adcd7862 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:19.392872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321d96f-1f71-46cb-9680-866634fd5ff3 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:19.401712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 307/607] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:37:19 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/1bd66c1b7c7147a281ef2e7253224069/roles/01737d3768e2407780ff51b345e1d020 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:37:19.403252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c12de-2bce-4ce1-89af-f689adcd7862 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:19.434084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d1f47-a621-4940-8a7d-9c4cc7f3fc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:19.435488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d1f47-a621-4940-8a7d-9c4cc7f3fc55 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:19.436075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d1f47-a621-4940-8a7d-9c4cc7f3fc55 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:19.436361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d1f47-a621-4940-8a7d-9c4cc7f3fc55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:19.451340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d1f47-a621-4940-8a7d-9c4cc7f3fc55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:19.458768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d1f47-a621-4940-8a7d-9c4cc7f3fc55 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-c61d1f47-a621-4940-8a7d-9c4cc7f3fc55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:19.470766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:19.471900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:19.473042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:19.473042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:19.473042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:19.473042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:19.473042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:19.473042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:19.473042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:19.473042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:19.473042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 301/608] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:19.487796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d1f47-a621-4940-8a7d-9c4cc7f3fc55 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:19.548693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:19.558691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c12de-2bce-4ce1-89af-f689adcd7862 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:19.604628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6a6b-52e6-45e4-9d19-1227f32c6c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:19.605310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6a6b-52e6-45e4-9d19-1227f32c6c64 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:19.605597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6a6b-52e6-45e4-9d19-1227f32c6c64 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:19.606348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6a6b-52e6-45e4-9d19-1227f32c6c64 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:19.607148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6a6b-52e6-45e4-9d19-1227f32c6c64 None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:19.607463 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6a6b-52e6-45e4-9d19-1227f32c6c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:19.609531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6a6b-52e6-45e4-9d19-1227f32c6c64 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:19.627738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:19.638594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 308/609] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:19.647056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6a6b-52e6-45e4-9d19-1227f32c6c64 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:19.718688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d1f47-a621-4940-8a7d-9c4cc7f3fc55 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:19.765756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18471f22-1a7b-4193-8626-3ecc249af00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:19.766691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18471f22-1a7b-4193-8626-3ecc249af00f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:19.767026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18471f22-1a7b-4193-8626-3ecc249af00f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:19.767305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18471f22-1a7b-4193-8626-3ecc249af00f None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:19.768591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18471f22-1a7b-4193-8626-3ecc249af00f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:19.768916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18471f22-1a7b-4193-8626-3ecc249af00f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:19.771082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18471f22-1a7b-4193-8626-3ecc249af00f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:19.778325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6a6b-52e6-45e4-9d19-1227f32c6c64 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:19.786722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 302/610] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:19.793458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18471f22-1a7b-4193-8626-3ecc249af00f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:19.856483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd666de7-3d8d-40ef-ba88-01337943b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:19.857355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd666de7-3d8d-40ef-ba88-01337943b42d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:19.857644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd666de7-3d8d-40ef-ba88-01337943b42d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:19.857920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd666de7-3d8d-40ef-ba88-01337943b42d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:19.878946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd666de7-3d8d-40ef-ba88-01337943b42d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:19.882378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd666de7-3d8d-40ef-ba88-01337943b42d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-dd666de7-3d8d-40ef-ba88-01337943b42d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:19.919179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:19.920139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:19.921014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:19.921014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:19.921014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:19.921014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:19.921014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 309/611] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:19.926273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd666de7-3d8d-40ef-ba88-01337943b42d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:19.959278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:19.969708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18471f22-1a7b-4193-8626-3ecc249af00f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:20.008108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:20.012614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7711ede-f15a-4b2c-85f0-b593e2a9d083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:20.013739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7711ede-f15a-4b2c-85f0-b593e2a9d083 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:20.014194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7711ede-f15a-4b2c-85f0-b593e2a9d083 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:20.015265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7711ede-f15a-4b2c-85f0-b593e2a9d083 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/bf1915b9919442d692bcedf25f53320c/roles/01737d3768e2407780ff51b345e1d020` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:20.032387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7711ede-f15a-4b2c-85f0-b593e2a9d083 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=bf1915b9919442d692bcedf25f53320c, role_id=01737d3768e2407780ff51b345e1d020)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:20.034218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7711ede-f15a-4b2c-85f0-b593e2a9d083 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:20.050232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 303/612] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:37:19 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/bf1915b9919442d692bcedf25f53320c/roles/01737d3768e2407780ff51b345e1d020 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:37:20.139051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd666de7-3d8d-40ef-ba88-01337943b42d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:20.191262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac443aa5-77ae-4b16-a8ac-c0d7d848e0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:20.191953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac443aa5-77ae-4b16-a8ac-c0d7d848e0d8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:20.192237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac443aa5-77ae-4b16-a8ac-c0d7d848e0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:20.192649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac443aa5-77ae-4b16-a8ac-c0d7d848e0d8 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:20.193394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac443aa5-77ae-4b16-a8ac-c0d7d848e0d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:20.193724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac443aa5-77ae-4b16-a8ac-c0d7d848e0d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:20.195826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac443aa5-77ae-4b16-a8ac-c0d7d848e0d8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:20.216195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 310/613] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:20.222086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7711ede-f15a-4b2c-85f0-b593e2a9d083 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:20.333474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7711ede-f15a-4b2c-85f0-b593e2a9d083 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:20.372777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ed6ea-e816-41c3-ae92-a50ff6098b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:20.374072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed6ea-e816-41c3-ae92-a50ff6098b0c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:20.374608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed6ea-e816-41c3-ae92-a50ff6098b0c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:20.375129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed6ea-e816-41c3-ae92-a50ff6098b0c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:20.386602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ed6ea-e816-41c3-ae92-a50ff6098b0c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:20.389671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ed6ea-e816-41c3-ae92-a50ff6098b0c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-d91ed6ea-e816-41c3-ae92-a50ff6098b0c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:20.410189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:20.411239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:20.412205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:20.415547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 304/614] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:20.421372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac443aa5-77ae-4b16-a8ac-c0d7d848e0d8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:20.497848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:20.533555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac443aa5-77ae-4b16-a8ac-c0d7d848e0d8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:20.569592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644290b-3290-440d-a974-b7f9dc1fec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:20.570350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644290b-3290-440d-a974-b7f9dc1fec50 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:20.570551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644290b-3290-440d-a974-b7f9dc1fec50 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:20.570863 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644290b-3290-440d-a974-b7f9dc1fec50 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:20.571702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644290b-3290-440d-a974-b7f9dc1fec50 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:20.572016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644290b-3290-440d-a974-b7f9dc1fec50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:20.573467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644290b-3290-440d-a974-b7f9dc1fec50 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:20.592103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 311/615] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Jun 15 15:37:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:20.597530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ed6ea-e816-41c3-ae92-a50ff6098b0c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:20.617404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:20.701243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ed6ea-e816-41c3-ae92-a50ff6098b0c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:20.736299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d5092-70cc-4e7e-8a89-98e821019f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:20.737103 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d5092-70cc-4e7e-8a89-98e821019f8e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:20.737436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d5092-70cc-4e7e-8a89-98e821019f8e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:20.737719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d5092-70cc-4e7e-8a89-98e821019f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:20.748863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d5092-70cc-4e7e-8a89-98e821019f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:20.750357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d5092-70cc-4e7e-8a89-98e821019f8e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-ec7d5092-70cc-4e7e-8a89-98e821019f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:20.768437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:20.769333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:20.770214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:20.770214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:20.770214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:20.770214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:20.770214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 305/616] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:20.775800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644290b-3290-440d-a974-b7f9dc1fec50 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:20.924901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644290b-3290-440d-a974-b7f9dc1fec50 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:20.982127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:21.006180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0ecf-b2f4-4d88-8c3d-050dd246a5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:21.006926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0ecf-b2f4-4d88-8c3d-050dd246a5af None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:21.007209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0ecf-b2f4-4d88-8c3d-050dd246a5af None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:21.007483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0ecf-b2f4-4d88-8c3d-050dd246a5af None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:21.008206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf0ecf-b2f4-4d88-8c3d-050dd246a5af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:21.008509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf0ecf-b2f4-4d88-8c3d-050dd246a5af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:21.009882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf0ecf-b2f4-4d88-8c3d-050dd246a5af None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:21.030172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 312/617] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:21.035341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d5092-70cc-4e7e-8a89-98e821019f8e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:21.143170 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:21.156728 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d5092-70cc-4e7e-8a89-98e821019f8e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:21.200420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b61b6-96e3-4c45-af13-e714828aa032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:21.201274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b61b6-96e3-4c45-af13-e714828aa032 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:21.201557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b61b6-96e3-4c45-af13-e714828aa032 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:21.201882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b61b6-96e3-4c45-af13-e714828aa032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:21.209517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b61b6-96e3-4c45-af13-e714828aa032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:21.211026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b61b6-96e3-4c45-af13-e714828aa032 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-da6b61b6-96e3-4c45-af13-e714828aa032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:21.228903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:21.229888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:21.230941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:21.230941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:21.230941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:21.230941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:21.230941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:21.230941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:21.230941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:21.230941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:21.235247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 306/618] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:21.243846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0ecf-b2f4-4d88-8c3d-050dd246a5af None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:21.401512 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0ecf-b2f4-4d88-8c3d-050dd246a5af None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:21.449049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d47ed-01e8-41e8-a2a1-61c5814b58d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:21.449525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d47ed-01e8-41e8-a2a1-61c5814b58d7 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:21.449812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d47ed-01e8-41e8-a2a1-61c5814b58d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:21.450088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d47ed-01e8-41e8-a2a1-61c5814b58d7 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:21.450906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d47ed-01e8-41e8-a2a1-61c5814b58d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:21.451215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d47ed-01e8-41e8-a2a1-61c5814b58d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:21.452650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d47ed-01e8-41e8-a2a1-61c5814b58d7 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:21.465830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 313/619] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:21.471346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b61b6-96e3-4c45-af13-e714828aa032 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:21.632766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b61b6-96e3-4c45-af13-e714828aa032 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:21.662291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:21.668895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa7594-ebba-4cf6-9c56-4540506d7ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:21.670390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa7594-ebba-4cf6-9c56-4540506d7ce0 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:21.675400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa7594-ebba-4cf6-9c56-4540506d7ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:21.675762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa7594-ebba-4cf6-9c56-4540506d7ce0 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/a6e3bb6553264387936e9a534afc3ac0/roles/01737d3768e2407780ff51b345e1d020` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:21.700749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa7594-ebba-4cf6-9c56-4540506d7ce0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=a6e3bb6553264387936e9a534afc3ac0, role_id=01737d3768e2407780ff51b345e1d020)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:21.702687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa7594-ebba-4cf6-9c56-4540506d7ce0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:21.735777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 307/620] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:37:21 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/a6e3bb6553264387936e9a534afc3ac0/roles/01737d3768e2407780ff51b345e1d020 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:37:21.743547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd622-22b1-45d6-aa44-8b456a814793 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:21.744132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd622-22b1-45d6-aa44-8b456a814793 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:21.744132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd622-22b1-45d6-aa44-8b456a814793 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:21.746854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 314/621] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:21.751893 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77b6f6-397c-49e1-9b3e-3b19c1e30ab1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:21.752194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77b6f6-397c-49e1-9b3e-3b19c1e30ab1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:21.752453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77b6f6-397c-49e1-9b3e-3b19c1e30ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:21.770189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-3b77b6f6-397c-49e1-9b3e-3b19c1e30ab1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:21.771717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3b77b6f6-397c-49e1-9b3e-3b19c1e30ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:21.804077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff51f8-2aab-4d1c-9061-661dfea07ad0 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:21.804417 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff51f8-2aab-4d1c-9061-661dfea07ad0 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:21.804632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff51f8-2aab-4d1c-9061-661dfea07ad0 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:21.806324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 315/622] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:21.811893 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54e575-8642-4534-be8a-0c5f27275320 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:21.812217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54e575-8642-4534-be8a-0c5f27275320 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:21.812453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54e575-8642-4534-be8a-0c5f27275320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:21.840920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-bf54e575-8642-4534-be8a-0c5f27275320 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:21.842149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bf54e575-8642-4534-be8a-0c5f27275320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:21.885922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 308/623] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:21.923196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:21.924165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77b6f6-397c-49e1-9b3e-3b19c1e30ab1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:21.946383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 316/624] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:21.984487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54e575-8642-4534-be8a-0c5f27275320 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.060204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77b6f6-397c-49e1-9b3e-3b19c1e30ab1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:22.072355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:22.099067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e5ae-d072-49d3-8147-3ded4b0a7f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:22.099908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e5ae-d072-49d3-8147-3ded4b0a7f75 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.100201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e5ae-d072-49d3-8147-3ded4b0a7f75 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.100663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e5ae-d072-49d3-8147-3ded4b0a7f75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.102526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54e575-8642-4534-be8a-0c5f27275320 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:22.112583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9e5ae-d072-49d3-8147-3ded4b0a7f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-20e9e5ae-d072-49d3-8147-3ded4b0a7f75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:22.121614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:22.122700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:22.123644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 309/625] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:22.132538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e5ae-d072-49d3-8147-3ded4b0a7f75 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.157376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0757402-9fd6-41b8-b948-e28b16a40e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:22.158091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0757402-9fd6-41b8-b948-e28b16a40e73 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.158368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0757402-9fd6-41b8-b948-e28b16a40e73 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.158638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0757402-9fd6-41b8-b948-e28b16a40e73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.179756 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0757402-9fd6-41b8-b948-e28b16a40e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-a0757402-9fd6-41b8-b948-e28b16a40e73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:22.187476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:22.188818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:22.189836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 317/626] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:22.191593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840c714-9941-4537-a92d-ac299bde3c13 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.191593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840c714-9941-4537-a92d-ac299bde3c13 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.192273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840c714-9941-4537-a92d-ac299bde3c13 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.194361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 318/627] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:22.196717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840c714-9941-4537-a92d-ac299bde3c13 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.238784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:22.288905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e5ae-d072-49d3-8147-3ded4b0a7f75 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:22.323279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840c714-9941-4537-a92d-ac299bde3c13 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:22.326207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb13571-e8ea-4ff6-946d-19bc8c4e725a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:22.327185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb13571-e8ea-4ff6-946d-19bc8c4e725a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.327691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb13571-e8ea-4ff6-946d-19bc8c4e725a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.328176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb13571-e8ea-4ff6-946d-19bc8c4e725a None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.329129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb13571-e8ea-4ff6-946d-19bc8c4e725a None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:22.329645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb13571-e8ea-4ff6-946d-19bc8c4e725a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:22.331349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb13571-e8ea-4ff6-946d-19bc8c4e725a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:22.341524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 310/628] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:22.343295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455a138-bd12-4822-af24-ae3ca0f718c5 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.343791 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455a138-bd12-4822-af24-ae3ca0f718c5 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.344287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455a138-bd12-4822-af24-ae3ca0f718c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.362873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-1455a138-bd12-4822-af24-ae3ca0f718c5 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:22.364564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1455a138-bd12-4822-af24-ae3ca0f718c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:22.367296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e44075-5334-4633-ac82-bd0a46e34c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:22.367971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e44075-5334-4633-ac82-bd0a46e34c77 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.369203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e44075-5334-4633-ac82-bd0a46e34c77 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.369203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e44075-5334-4633-ac82-bd0a46e34c77 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.369393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e44075-5334-4633-ac82-bd0a46e34c77 None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:22.369584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e44075-5334-4633-ac82-bd0a46e34c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:22.371917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e44075-5334-4633-ac82-bd0a46e34c77 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:22.382586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 319/629] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:22.383461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e44075-5334-4633-ac82-bd0a46e34c77 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.399024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:22.458836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 311/630] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:22.460298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455a138-bd12-4822-af24-ae3ca0f718c5 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.520947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:22.525133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e44075-5334-4633-ac82-bd0a46e34c77 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:22.572589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5096df-01ff-4823-9ae2-02c62903f1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:22.573393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5096df-01ff-4823-9ae2-02c62903f1cf None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.573676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5096df-01ff-4823-9ae2-02c62903f1cf None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.573947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5096df-01ff-4823-9ae2-02c62903f1cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.589368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5096df-01ff-4823-9ae2-02c62903f1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:22.589820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455a138-bd12-4822-af24-ae3ca0f718c5 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:22.590872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5096df-01ff-4823-9ae2-02c62903f1cf None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-df5096df-01ff-4823-9ae2-02c62903f1cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:22.613431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:22.614390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:22.615343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:22.615343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:22.615343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:22.615343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:22.618866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 320/631] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:22.620354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5096df-01ff-4823-9ae2-02c62903f1cf None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.645074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86065b0b-2aaa-4914-8088-214422899bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:22.646570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86065b0b-2aaa-4914-8088-214422899bc9 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.647099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86065b0b-2aaa-4914-8088-214422899bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.647874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86065b0b-2aaa-4914-8088-214422899bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.661987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86065b0b-2aaa-4914-8088-214422899bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:22.664489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86065b0b-2aaa-4914-8088-214422899bc9 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-86065b0b-2aaa-4914-8088-214422899bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:22.685151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:22.686139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:22.687053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:22.687053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:22.687053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:22.687053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:22.689395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 312/632] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:22.690542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86065b0b-2aaa-4914-8088-214422899bc9 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.707292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:22.731655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5096df-01ff-4823-9ae2-02c62903f1cf None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:22.769246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911d5fc-0b3d-412b-9dba-3ed10cb92b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:22.769948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911d5fc-0b3d-412b-9dba-3ed10cb92b8b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.769948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911d5fc-0b3d-412b-9dba-3ed10cb92b8b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.770263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911d5fc-0b3d-412b-9dba-3ed10cb92b8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.782634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911d5fc-0b3d-412b-9dba-3ed10cb92b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-2911d5fc-0b3d-412b-9dba-3ed10cb92b8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:22.790823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:22.791731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:22.792586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:22.796848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 321/633] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:22.797689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911d5fc-0b3d-412b-9dba-3ed10cb92b8b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.810676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:22.812675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86065b0b-2aaa-4914-8088-214422899bc9 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:22.848491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73a8e4-ffb4-435f-8119-588e47846fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:22.849155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73a8e4-ffb4-435f-8119-588e47846fdc None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.849436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73a8e4-ffb4-435f-8119-588e47846fdc None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.849715 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73a8e4-ffb4-435f-8119-588e47846fdc None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.851120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73a8e4-ffb4-435f-8119-588e47846fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:22.851435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73a8e4-ffb4-435f-8119-588e47846fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:22.852803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73a8e4-ffb4-435f-8119-588e47846fdc None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:22.869203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 313/634] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:22.869203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73a8e4-ffb4-435f-8119-588e47846fdc None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.904134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911d5fc-0b3d-412b-9dba-3ed10cb92b8b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:22.936374 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e38779-ccbf-40d7-9a13-6146930d6323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:22.937099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e38779-ccbf-40d7-9a13-6146930d6323 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.937438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e38779-ccbf-40d7-9a13-6146930d6323 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.937798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e38779-ccbf-40d7-9a13-6146930d6323 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.938574 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e38779-ccbf-40d7-9a13-6146930d6323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:22.938975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e38779-ccbf-40d7-9a13-6146930d6323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:22.940425 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e38779-ccbf-40d7-9a13-6146930d6323 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:22.945519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:22.956468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 322/635] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:22.956468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f0252-a24e-4118-b4d9-6c743f0953fb None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:22.956468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f0252-a24e-4118-b4d9-6c743f0953fb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:22.956468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f0252-a24e-4118-b4d9-6c743f0953fb None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:22.956468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 323/636] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:22.956468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f0252-a24e-4118-b4d9-6c743f0953fb None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:22.981336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73a8e4-ffb4-435f-8119-588e47846fdc None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.016818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b413cbd-3b03-4b78-af49-8f6ad14ef9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.017659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b413cbd-3b03-4b78-af49-8f6ad14ef9b9 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.018062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b413cbd-3b03-4b78-af49-8f6ad14ef9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.018439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b413cbd-3b03-4b78-af49-8f6ad14ef9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.019334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b413cbd-3b03-4b78-af49-8f6ad14ef9b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:23.019732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b413cbd-3b03-4b78-af49-8f6ad14ef9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:23.021434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b413cbd-3b03-4b78-af49-8f6ad14ef9b9 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:23.030266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 314/637] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:23.031011 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b413cbd-3b03-4b78-af49-8f6ad14ef9b9 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:23.037452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:23.070788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f0252-a24e-4118-b4d9-6c743f0953fb None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.105692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682a422-6bc4-4ea5-9b76-96923db49fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.106572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682a422-6bc4-4ea5-9b76-96923db49fb0 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.106951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682a422-6bc4-4ea5-9b76-96923db49fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.107383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682a422-6bc4-4ea5-9b76-96923db49fb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.117527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682a422-6bc4-4ea5-9b76-96923db49fb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:23.119032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682a422-6bc4-4ea5-9b76-96923db49fb0 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-3682a422-6bc4-4ea5-9b76-96923db49fb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:23.133632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:23.134926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:23.135930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 324/638] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Jun 15 15:37:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:23.136543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b4164-2db8-4374-9500-9a6d5ad30b5f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.137039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b4164-2db8-4374-9500-9a6d5ad30b5f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.137039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b4164-2db8-4374-9500-9a6d5ad30b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.146068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b413cbd-3b03-4b78-af49-8f6ad14ef9b9 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.154690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-946b4164-2db8-4374-9500-9a6d5ad30b5f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:23.156285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-946b4164-2db8-4374-9500-9a6d5ad30b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:23.184618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b76998-a409-4977-85e8-9624232540f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.185396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b76998-a409-4977-85e8-9624232540f9 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.185682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b76998-a409-4977-85e8-9624232540f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.185965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b76998-a409-4977-85e8-9624232540f9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.189323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:23.195966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b76998-a409-4977-85e8-9624232540f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:23.197997 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b76998-a409-4977-85e8-9624232540f9 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-f1b76998-a409-4977-85e8-9624232540f9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:23.213032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:23.214060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 315/639] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:23.215561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b76998-a409-4977-85e8-9624232540f9 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:23.231543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 325/640] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:23.235973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b4164-2db8-4374-9500-9a6d5ad30b5f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:23.310318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:23.328597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b76998-a409-4977-85e8-9624232540f9 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.353449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b4164-2db8-4374-9500-9a6d5ad30b5f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.365163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd05a2-932e-4de0-a9a7-f424ebd94664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.365992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd05a2-932e-4de0-a9a7-f424ebd94664 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.366278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd05a2-932e-4de0-a9a7-f424ebd94664 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.366559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd05a2-932e-4de0-a9a7-f424ebd94664 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.377352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd05a2-932e-4de0-a9a7-f424ebd94664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:23.378789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd05a2-932e-4de0-a9a7-f424ebd94664 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:23.390340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52867f60-8231-46a7-a189-cd3ab6c53efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.391214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52867f60-8231-46a7-a189-cd3ab6c53efe None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.391666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52867f60-8231-46a7-a189-cd3ab6c53efe None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.392093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52867f60-8231-46a7-a189-cd3ab6c53efe None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.393067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52867f60-8231-46a7-a189-cd3ab6c53efe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:23.393525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52867f60-8231-46a7-a189-cd3ab6c53efe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:23.395117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52867f60-8231-46a7-a189-cd3ab6c53efe None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-e9fd05a2-932e-4de0-a9a7-f424ebd94664 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:23.396730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:23.398400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:23.399437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:23.399437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:23.399437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:23.399437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:23.399437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 316/641] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:23.399742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd05a2-932e-4de0-a9a7-f424ebd94664 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:23.420554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 326/642] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Jun 15 15:37:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:23.421444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52867f60-8231-46a7-a189-cd3ab6c53efe None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:23.464699 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:23.519447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd05a2-932e-4de0-a9a7-f424ebd94664 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.539435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52867f60-8231-46a7-a189-cd3ab6c53efe None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.557455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928de1b-3fd9-4ecf-8567-16f21a62a17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.557914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:23.558121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928de1b-3fd9-4ecf-8567-16f21a62a17f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.558408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928de1b-3fd9-4ecf-8567-16f21a62a17f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.558695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928de1b-3fd9-4ecf-8567-16f21a62a17f None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.559963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928de1b-3fd9-4ecf-8567-16f21a62a17f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:23.560304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928de1b-3fd9-4ecf-8567-16f21a62a17f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:23.561810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928de1b-3fd9-4ecf-8567-16f21a62a17f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:23.576609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d6a1c-df82-4d51-850e-8cc2b2b2199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.577403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6a1c-df82-4d51-850e-8cc2b2b2199c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.577770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6a1c-df82-4d51-850e-8cc2b2b2199c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.577965 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6a1c-df82-4d51-850e-8cc2b2b2199c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.580213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 317/643] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Jun 15 15:37:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:23.581213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928de1b-3fd9-4ecf-8567-16f21a62a17f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:23.594204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d6a1c-df82-4d51-850e-8cc2b2b2199c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-f74d6a1c-df82-4d51-850e-8cc2b2b2199c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:23.602433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:23.603429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:23.604338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:23.608518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 327/644] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:23.609382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d6a1c-df82-4d51-850e-8cc2b2b2199c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:23.691856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928de1b-3fd9-4ecf-8567-16f21a62a17f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.700112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:23.714808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d6a1c-df82-4d51-850e-8cc2b2b2199c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.730264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f8e79-6442-44f2-97eb-84c77e789243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.730966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f8e79-6442-44f2-97eb-84c77e789243 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.731251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f8e79-6442-44f2-97eb-84c77e789243 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.731536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f8e79-6442-44f2-97eb-84c77e789243 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.732260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f8e79-6442-44f2-97eb-84c77e789243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:23.732572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f8e79-6442-44f2-97eb-84c77e789243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:23.733926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f8e79-6442-44f2-97eb-84c77e789243 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:23.745440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 318/645] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:23.746185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f8e79-6442-44f2-97eb-84c77e789243 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:23.756337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29b4c4-59df-462c-b0c7-181897346899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.757284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29b4c4-59df-462c-b0c7-181897346899 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.757762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29b4c4-59df-462c-b0c7-181897346899 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.758224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29b4c4-59df-462c-b0c7-181897346899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.809699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:23.833515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29b4c4-59df-462c-b0c7-181897346899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:23.835598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29b4c4-59df-462c-b0c7-181897346899 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:23.853511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f8e79-6442-44f2-97eb-84c77e789243 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:23.887655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366c277-d7d4-469f-8784-18aa39b2eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:23.888304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366c277-d7d4-469f-8784-18aa39b2eba4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:23.888589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366c277-d7d4-469f-8784-18aa39b2eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:23.888869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366c277-d7d4-469f-8784-18aa39b2eba4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:23.924946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:23.949195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 328/646] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:23.950037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29b4c4-59df-462c-b0c7-181897346899 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:23.966539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366c277-d7d4-469f-8784-18aa39b2eba4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:23.968054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366c277-d7d4-469f-8784-18aa39b2eba4 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.038625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:24.066554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29b4c4-59df-462c-b0c7-181897346899 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.080708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 319/647] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:24.080708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366c277-d7d4-469f-8784-18aa39b2eba4 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.108025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a7b67-3a2e-4bc1-8caa-6759fa8e1aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.108694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a7b67-3a2e-4bc1-8caa-6759fa8e1aaa None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.108976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a7b67-3a2e-4bc1-8caa-6759fa8e1aaa None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.109251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a7b67-3a2e-4bc1-8caa-6759fa8e1aaa None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.109981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a7b67-3a2e-4bc1-8caa-6759fa8e1aaa None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:24.110285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a7b67-3a2e-4bc1-8caa-6759fa8e1aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.111672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a7b67-3a2e-4bc1-8caa-6759fa8e1aaa None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.120175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 329/648] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:24.120786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a7b67-3a2e-4bc1-8caa-6759fa8e1aaa None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.147822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:24.183673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366c277-d7d4-469f-8784-18aa39b2eba4 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.216568 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06108480-c812-434a-bec9-ad313ac90c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.217349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06108480-c812-434a-bec9-ad313ac90c7c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.217476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a7b67-3a2e-4bc1-8caa-6759fa8e1aaa None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.217629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06108480-c812-434a-bec9-ad313ac90c7c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.217903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06108480-c812-434a-bec9-ad313ac90c7c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.225739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06108480-c812-434a-bec9-ad313ac90c7c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.227205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06108480-c812-434a-bec9-ad313ac90c7c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-06108480-c812-434a-bec9-ad313ac90c7c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:24.241569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:24.242637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 320/649] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:24.243799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06108480-c812-434a-bec9-ad313ac90c7c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.254661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69f1ba-c943-452f-908f-1b080c3f378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.255483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69f1ba-c943-452f-908f-1b080c3f378b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.255769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69f1ba-c943-452f-908f-1b080c3f378b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.256050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69f1ba-c943-452f-908f-1b080c3f378b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.264643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:24.267299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69f1ba-c943-452f-908f-1b080c3f378b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.269057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69f1ba-c943-452f-908f-1b080c3f378b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-7c69f1ba-c943-452f-908f-1b080c3f378b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:24.286314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:24.287213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:24.288624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:24.288624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:24.288624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:24.288624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:24.288624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 330/650] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:24.289216 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355147a-c33f-465b-b131-e805043a45de None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.289500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355147a-c33f-465b-b131-e805043a45de None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.289782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355147a-c33f-465b-b131-e805043a45de None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.291088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 331/651] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:24.291782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355147a-c33f-465b-b131-e805043a45de None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.345317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06108480-c812-434a-bec9-ad313ac90c7c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.370796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:24.379828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07a41a-d9b2-451d-b2a3-1a508ee07b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.380613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07a41a-d9b2-451d-b2a3-1a508ee07b0f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.380895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07a41a-d9b2-451d-b2a3-1a508ee07b0f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.381169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07a41a-d9b2-451d-b2a3-1a508ee07b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.391343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355147a-c33f-465b-b131-e805043a45de None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.392218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07a41a-d9b2-451d-b2a3-1a508ee07b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.393648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07a41a-d9b2-451d-b2a3-1a508ee07b0f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-4d07a41a-d9b2-451d-b2a3-1a508ee07b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:24.410332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:24.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:24.412313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:24.412313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:24.412313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:24.412313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:24.412313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 321/652] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:24.412313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07a41a-d9b2-451d-b2a3-1a508ee07b0f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.429304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e976a-ec5f-4b90-9015-5fd9241aacfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.430092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e976a-ec5f-4b90-9015-5fd9241aacfd None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.430375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e976a-ec5f-4b90-9015-5fd9241aacfd None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.430754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e976a-ec5f-4b90-9015-5fd9241aacfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.445153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e976a-ec5f-4b90-9015-5fd9241aacfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.446650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e976a-ec5f-4b90-9015-5fd9241aacfd None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-1d1e976a-ec5f-4b90-9015-5fd9241aacfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:24.465392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:24.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:24.467677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:24.467677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:24.467677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:24.467677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:24.467677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 332/653] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:24.467677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e976a-ec5f-4b90-9015-5fd9241aacfd None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.511832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:24.526424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07a41a-d9b2-451d-b2a3-1a508ee07b0f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.559980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c373d36-1c60-4707-ba66-321e7d5dfc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.560645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373d36-1c60-4707-ba66-321e7d5dfc49 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.560929 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373d36-1c60-4707-ba66-321e7d5dfc49 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.561208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373d36-1c60-4707-ba66-321e7d5dfc49 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.562039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c373d36-1c60-4707-ba66-321e7d5dfc49 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:24.562351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c373d36-1c60-4707-ba66-321e7d5dfc49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.563815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c373d36-1c60-4707-ba66-321e7d5dfc49 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.564983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e976a-ec5f-4b90-9015-5fd9241aacfd None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.580391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 322/654] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:24.581578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320f0f2-e06e-465b-962e-2b3ad8ceea0f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.581867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320f0f2-e06e-465b-962e-2b3ad8ceea0f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.582145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320f0f2-e06e-465b-962e-2b3ad8ceea0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.598144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d320f0f2-e06e-465b-962e-2b3ad8ceea0f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:24.599297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d320f0f2-e06e-465b-962e-2b3ad8ceea0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:24.602456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5559e5-9980-4638-8c0a-0ad9cfc57e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.603138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5559e5-9980-4638-8c0a-0ad9cfc57e09 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.603424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5559e5-9980-4638-8c0a-0ad9cfc57e09 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.603702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5559e5-9980-4638-8c0a-0ad9cfc57e09 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.604417 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5559e5-9980-4638-8c0a-0ad9cfc57e09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:24.604725 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5559e5-9980-4638-8c0a-0ad9cfc57e09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.606079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5559e5-9980-4638-8c0a-0ad9cfc57e09 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.617503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 333/655] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:24.618276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5559e5-9980-4638-8c0a-0ad9cfc57e09 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.627393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:24.666370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 323/656] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:24.667222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320f0f2-e06e-465b-962e-2b3ad8ceea0f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.721613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5559e5-9980-4638-8c0a-0ad9cfc57e09 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.723888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:24.754600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130893c-d666-4228-b0ee-f1b13f8c3401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.755292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130893c-d666-4228-b0ee-f1b13f8c3401 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.755573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130893c-d666-4228-b0ee-f1b13f8c3401 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.755847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130893c-d666-4228-b0ee-f1b13f8c3401 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.757077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130893c-d666-4228-b0ee-f1b13f8c3401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:24.757378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130893c-d666-4228-b0ee-f1b13f8c3401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.758758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130893c-d666-4228-b0ee-f1b13f8c3401 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.769606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 334/657] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:24.769735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320f0f2-e06e-465b-962e-2b3ad8ceea0f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.770635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130893c-d666-4228-b0ee-f1b13f8c3401 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.804562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6bf1e-7354-43a7-bc52-b21389c9945a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.805215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6bf1e-7354-43a7-bc52-b21389c9945a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.805499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6bf1e-7354-43a7-bc52-b21389c9945a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.805781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6bf1e-7354-43a7-bc52-b21389c9945a None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.806498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6bf1e-7354-43a7-bc52-b21389c9945a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:24.806831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6bf1e-7354-43a7-bc52-b21389c9945a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.808182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6bf1e-7354-43a7-bc52-b21389c9945a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.819167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 324/658] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:24.820108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6bf1e-7354-43a7-bc52-b21389c9945a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:24.846431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:24.873754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130893c-d666-4228-b0ee-f1b13f8c3401 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.906261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4f734-d09b-4e83-b0c9-9c480c3baf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.906930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4f734-d09b-4e83-b0c9-9c480c3baf79 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.907212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4f734-d09b-4e83-b0c9-9c480c3baf79 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.907488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4f734-d09b-4e83-b0c9-9c480c3baf79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.917175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6bf1e-7354-43a7-bc52-b21389c9945a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:24.937766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:24.950540 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3513d8-1ee1-4a64-abb7-6561f9d8b73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:24.951354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3513d8-1ee1-4a64-abb7-6561f9d8b73e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:24.951647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3513d8-1ee1-4a64-abb7-6561f9d8b73e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:24.951922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3513d8-1ee1-4a64-abb7-6561f9d8b73e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:24.958802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3513d8-1ee1-4a64-abb7-6561f9d8b73e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.960240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3513d8-1ee1-4a64-abb7-6561f9d8b73e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.973553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4f734-d09b-4e83-b0c9-9c480c3baf79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-fb3513d8-1ee1-4a64-abb7-6561f9d8b73e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:24.974782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:24.976357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:24.977277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:24.977277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:24.977277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:24.977277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:24.977277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:24.977277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:24.977277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:24.977277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:24.977897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4f734-d09b-4e83-b0c9-9c480c3baf79 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:24.977971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 325/659] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:24.977971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3513d8-1ee1-4a64-abb7-6561f9d8b73e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.059202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:25.073977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3513d8-1ee1-4a64-abb7-6561f9d8b73e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.077051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 335/660] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:25.077872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4f734-d09b-4e83-b0c9-9c480c3baf79 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.107871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbd5c2-e191-4ff9-96c3-ae956ac36dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.108645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbd5c2-e191-4ff9-96c3-ae956ac36dd4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.108924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbd5c2-e191-4ff9-96c3-ae956ac36dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.109200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbd5c2-e191-4ff9-96c3-ae956ac36dd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.119721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbd5c2-e191-4ff9-96c3-ae956ac36dd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-5ebbd5c2-e191-4ff9-96c3-ae956ac36dd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:25.127333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:25.128435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 326/661] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:25.129662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbd5c2-e191-4ff9-96c3-ae956ac36dd4 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.144622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:25.182119 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4f734-d09b-4e83-b0c9-9c480c3baf79 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.216888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd937aaa-9655-421e-bf96-e4e7e7206110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.217672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937aaa-9655-421e-bf96-e4e7e7206110 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.217962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937aaa-9655-421e-bf96-e4e7e7206110 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.218234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd937aaa-9655-421e-bf96-e4e7e7206110 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.224808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd937aaa-9655-421e-bf96-e4e7e7206110 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.226243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd937aaa-9655-421e-bf96-e4e7e7206110 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.229830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbd5c2-e191-4ff9-96c3-ae956ac36dd4 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-dd937aaa-9655-421e-bf96-e4e7e7206110 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:25.239657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:25.241237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 336/662] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:25.242154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd937aaa-9655-421e-bf96-e4e7e7206110 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.266621 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f8057-7e41-42af-92a1-3fd3a618a198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.267441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f8057-7e41-42af-92a1-3fd3a618a198 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.267728 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f8057-7e41-42af-92a1-3fd3a618a198 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.268006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f8057-7e41-42af-92a1-3fd3a618a198 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.275652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f8057-7e41-42af-92a1-3fd3a618a198 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.277091 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f8057-7e41-42af-92a1-3fd3a618a198 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-b56f8057-7e41-42af-92a1-3fd3a618a198 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:25.291014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:25.292169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:25.293129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 327/663] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:25.293781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f8057-7e41-42af-92a1-3fd3a618a198 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.302885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:25.342321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd937aaa-9655-421e-bf96-e4e7e7206110 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.364880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:25.376604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ab5dd-1dce-4b4b-9062-6e888bd8e55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.377261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ab5dd-1dce-4b4b-9062-6e888bd8e55a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.377593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ab5dd-1dce-4b4b-9062-6e888bd8e55a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.377820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ab5dd-1dce-4b4b-9062-6e888bd8e55a None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.378594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ab5dd-1dce-4b4b-9062-6e888bd8e55a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:25.378933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ab5dd-1dce-4b4b-9062-6e888bd8e55a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.380352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ab5dd-1dce-4b4b-9062-6e888bd8e55a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.392759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 337/664] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:25.393571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ab5dd-1dce-4b4b-9062-6e888bd8e55a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.393857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f8057-7e41-42af-92a1-3fd3a618a198 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.429608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc721f38-7e2f-4666-9ebc-d0854c6ac11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.430386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc721f38-7e2f-4666-9ebc-d0854c6ac11e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.430670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc721f38-7e2f-4666-9ebc-d0854c6ac11e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.430974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc721f38-7e2f-4666-9ebc-d0854c6ac11e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.441358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc721f38-7e2f-4666-9ebc-d0854c6ac11e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.442812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc721f38-7e2f-4666-9ebc-d0854c6ac11e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-cc721f38-7e2f-4666-9ebc-d0854c6ac11e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:25.460343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:25.461212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:25.462106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:25.462106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:25.462106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:25.462106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:25.462106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 328/665] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:25.462374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc721f38-7e2f-4666-9ebc-d0854c6ac11e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.496724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ab5dd-1dce-4b4b-9062-6e888bd8e55a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.524439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:25.529538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd5b1a-8852-468f-84b2-ce81f6bcffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.530190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd5b1a-8852-468f-84b2-ce81f6bcffda None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.530466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd5b1a-8852-468f-84b2-ce81f6bcffda None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.530748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd5b1a-8852-468f-84b2-ce81f6bcffda None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.531486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd5b1a-8852-468f-84b2-ce81f6bcffda None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:25.531793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd5b1a-8852-468f-84b2-ce81f6bcffda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.533152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd5b1a-8852-468f-84b2-ce81f6bcffda None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.541580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 338/666] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:25.542170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd5b1a-8852-468f-84b2-ce81f6bcffda None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.571214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc721f38-7e2f-4666-9ebc-d0854c6ac11e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.586309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:25.604573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3a777-8236-4a8b-854a-1fbbbb408016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.605221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3a777-8236-4a8b-854a-1fbbbb408016 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.605500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3a777-8236-4a8b-854a-1fbbbb408016 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.605776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3a777-8236-4a8b-854a-1fbbbb408016 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.606550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3a777-8236-4a8b-854a-1fbbbb408016 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:25.606884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3a777-8236-4a8b-854a-1fbbbb408016 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.608292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3a777-8236-4a8b-854a-1fbbbb408016 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.619850 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 329/667] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:25.620624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3a777-8236-4a8b-854a-1fbbbb408016 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.638658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd5b1a-8852-468f-84b2-ce81f6bcffda None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.671986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd8efc-7a48-4b48-b559-379b734e091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.672675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd8efc-7a48-4b48-b559-379b734e091f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.672957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd8efc-7a48-4b48-b559-379b734e091f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.673230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd8efc-7a48-4b48-b559-379b734e091f None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.674053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd8efc-7a48-4b48-b559-379b734e091f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:25.674359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd8efc-7a48-4b48-b559-379b734e091f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.675817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd8efc-7a48-4b48-b559-379b734e091f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.693356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 339/668] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:25.694166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd8efc-7a48-4b48-b559-379b734e091f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.721158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3a777-8236-4a8b-854a-1fbbbb408016 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.738450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:25.755764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b808bb3-0476-48e3-a50a-e9fc3f8d0e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.756402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b808bb3-0476-48e3-a50a-e9fc3f8d0e06 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.756685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b808bb3-0476-48e3-a50a-e9fc3f8d0e06 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.756959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b808bb3-0476-48e3-a50a-e9fc3f8d0e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.757773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b808bb3-0476-48e3-a50a-e9fc3f8d0e06 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=4327e4eed5a1436ea1a00c2209bc5750) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:25.758080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b808bb3-0476-48e3-a50a-e9fc3f8d0e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.759517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b808bb3-0476-48e3-a50a-e9fc3f8d0e06 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.765125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 330/669] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/role_assignments?user.id=4327e4eed5a1436ea1a00c2209bc5750&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:25.765885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b808bb3-0476-48e3-a50a-e9fc3f8d0e06 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.802599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd8efc-7a48-4b48-b559-379b734e091f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.806549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:25.841677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33210be-7189-4cec-99c8-ce0865b0e7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.842360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33210be-7189-4cec-99c8-ce0865b0e7c2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.842632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33210be-7189-4cec-99c8-ce0865b0e7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.842940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33210be-7189-4cec-99c8-ce0865b0e7c2 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.843677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33210be-7189-4cec-99c8-ce0865b0e7c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:25.843992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33210be-7189-4cec-99c8-ce0865b0e7c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.845389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33210be-7189-4cec-99c8-ce0865b0e7c2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.859186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 340/670] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:25.860066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33210be-7189-4cec-99c8-ce0865b0e7c2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.883827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b808bb3-0476-48e3-a50a-e9fc3f8d0e06 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.921449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe4a4a-fa78-4122-a8a5-3cd302c8de81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:25.922196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe4a4a-fa78-4122-a8a5-3cd302c8de81 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:25.922482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe4a4a-fa78-4122-a8a5-3cd302c8de81 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:25.922920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe4a4a-fa78-4122-a8a5-3cd302c8de81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:25.936244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe4a4a-fa78-4122-a8a5-3cd302c8de81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:25.937104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe4a4a-fa78-4122-a8a5-3cd302c8de81 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-cdfe4a4a-fa78-4122-a8a5-3cd302c8de81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:25.954496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:25.957229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:25.958280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:25.958280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:25.958280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:25.958280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:25.958280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 331/671] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:25.958280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe4a4a-fa78-4122-a8a5-3cd302c8de81 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:25.980681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33210be-7189-4cec-99c8-ce0865b0e7c2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:25.999382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:26.021863 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366bc50-078d-4f65-bc2b-9adba642443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:26.022551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366bc50-078d-4f65-bc2b-9adba642443e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:26.022895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366bc50-078d-4f65-bc2b-9adba642443e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:26.023180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366bc50-078d-4f65-bc2b-9adba642443e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:26.024014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366bc50-078d-4f65-bc2b-9adba642443e None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=5f7405c8bc8e4c84bb056cd9e1df61c7) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:26.024328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366bc50-078d-4f65-bc2b-9adba642443e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:26.025765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366bc50-078d-4f65-bc2b-9adba642443e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:26.031719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 341/672] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/role_assignments?user.id=5f7405c8bc8e4c84bb056cd9e1df61c7&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:26.032503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366bc50-078d-4f65-bc2b-9adba642443e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:26.086292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe4a4a-fa78-4122-a8a5-3cd302c8de81 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:26.091134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:26.128123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d783fc-7170-4d01-bfda-f777b9354dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:26.128795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d783fc-7170-4d01-bfda-f777b9354dda None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:26.129080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d783fc-7170-4d01-bfda-f777b9354dda None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:26.129379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d783fc-7170-4d01-bfda-f777b9354dda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:26.177262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366bc50-078d-4f65-bc2b-9adba642443e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:26.225869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d783fc-7170-4d01-bfda-f777b9354dda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:26.226606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05aa24-8844-468f-970a-f964f36bb8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:26.227491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05aa24-8844-468f-970a-f964f36bb8b8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:26.227778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05aa24-8844-468f-970a-f964f36bb8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:26.228057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05aa24-8844-468f-970a-f964f36bb8b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:26.228224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d783fc-7170-4d01-bfda-f777b9354dda None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:26.235417 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05aa24-8844-468f-970a-f964f36bb8b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:26.236943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05aa24-8844-468f-970a-f964f36bb8b8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-0d05aa24-8844-468f-970a-f964f36bb8b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:26.254849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:26.255780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:26.256710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:26.256710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:26.256710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:26.256710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:26.256710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:26.256710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:26.256710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:26.256710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:26.256710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 342/673] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:26.257421 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05aa24-8844-468f-970a-f964f36bb8b8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:26.266437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:26.361692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 332/674] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:26.361692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d783fc-7170-4d01-bfda-f777b9354dda None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:26.378985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:26.395622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05aa24-8844-468f-970a-f964f36bb8b8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:26.444367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fe449-a605-47f5-a1bf-c6ba372433fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:26.445003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fe449-a605-47f5-a1bf-c6ba372433fa None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:26.445356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fe449-a605-47f5-a1bf-c6ba372433fa None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:26.449440 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fe449-a605-47f5-a1bf-c6ba372433fa None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:26.449440 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fe449-a605-47f5-a1bf-c6ba372433fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:26.449440 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fe449-a605-47f5-a1bf-c6ba372433fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:26.449440 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fe449-a605-47f5-a1bf-c6ba372433fa None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:26.464263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 343/675] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:26.465804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fe449-a605-47f5-a1bf-c6ba372433fa None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:26.498839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d783fc-7170-4d01-bfda-f777b9354dda None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:26.537287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff3d8e-38dc-4606-aba3-27afaf55bae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:26.538804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff3d8e-38dc-4606-aba3-27afaf55bae7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:26.539727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff3d8e-38dc-4606-aba3-27afaf55bae7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:26.540133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff3d8e-38dc-4606-aba3-27afaf55bae7 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:26.541295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff3d8e-38dc-4606-aba3-27afaf55bae7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:26.541716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff3d8e-38dc-4606-aba3-27afaf55bae7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:26.543298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff3d8e-38dc-4606-aba3-27afaf55bae7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:26.545341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:26.565723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 333/676] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:26.565723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff3d8e-38dc-4606-aba3-27afaf55bae7 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:26.587239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fe449-a605-47f5-a1bf-c6ba372433fa None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:26.622877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0798f8-2f4e-46b4-bf4f-d8491f716411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:26.623674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0798f8-2f4e-46b4-bf4f-d8491f716411 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:26.623952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0798f8-2f4e-46b4-bf4f-d8491f716411 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:26.624223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0798f8-2f4e-46b4-bf4f-d8491f716411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:26.635795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0798f8-2f4e-46b4-bf4f-d8491f716411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:26.637244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0798f8-2f4e-46b4-bf4f-d8491f716411 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:26.645740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-bc0798f8-2f4e-46b4-bf4f-d8491f716411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:26.657139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:26.658096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:26.659030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:26.659030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:26.659030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:26.659030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:26.665758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 344/677] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:26.666790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0798f8-2f4e-46b4-bf4f-d8491f716411 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:26.690043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff3d8e-38dc-4606-aba3-27afaf55bae7 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:26.732102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85525f-8c19-4578-a834-160185532cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:26.733254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85525f-8c19-4578-a834-160185532cdb None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:26.733679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85525f-8c19-4578-a834-160185532cdb None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:26.734107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85525f-8c19-4578-a834-160185532cdb None admin] PATH_INFO: `/v3/users/swift` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:26.742293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85525f-8c19-4578-a834-160185532cdb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:26.744135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85525f-8c19-4578-a834-160185532cdb None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-ab85525f-8c19-4578-a834-160185532cdb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:26.758902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:26.760067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:26.761163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:26.763707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 334/678] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:26.764848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85525f-8c19-4578-a834-160185532cdb None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:26.791121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0798f8-2f4e-46b4-bf4f-d8491f716411 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:26.826620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff534ac-de27-4ab2-8877-e45e1717f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:26.827301 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff534ac-de27-4ab2-8877-e45e1717f62f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:26.827584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff534ac-de27-4ab2-8877-e45e1717f62f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:26.828015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff534ac-de27-4ab2-8877-e45e1717f62f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:26.828845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff534ac-de27-4ab2-8877-e45e1717f62f None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=1bd66c1b7c7147a281ef2e7253224069) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:26.829156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff534ac-de27-4ab2-8877-e45e1717f62f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:26.829882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:26.831751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff534ac-de27-4ab2-8877-e45e1717f62f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:26.846756 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 345/679] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:26 2022] GET /identity/v3/role_assignments?user.id=1bd66c1b7c7147a281ef2e7253224069&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:26.846756 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff534ac-de27-4ab2-8877-e45e1717f62f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:26.910100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85525f-8c19-4578-a834-160185532cdb None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:26.911537 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:26.957540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff534ac-de27-4ab2-8877-e45e1717f62f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:26.963641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b722d4-8e65-425f-98e5-48881e69604f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:26.964432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b722d4-8e65-425f-98e5-48881e69604f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:26.964810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b722d4-8e65-425f-98e5-48881e69604f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:26.965160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b722d4-8e65-425f-98e5-48881e69604f None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:26.965987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b722d4-8e65-425f-98e5-48881e69604f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:26.966372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b722d4-8e65-425f-98e5-48881e69604f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:26.968482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b722d4-8e65-425f-98e5-48881e69604f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:26.999401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 335/680] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:27.000338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b722d4-8e65-425f-98e5-48881e69604f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:27.001497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90e1a9-e908-4c00-a580-730b79731389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:27.002831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90e1a9-e908-4c00-a580-730b79731389 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:27.003491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90e1a9-e908-4c00-a580-730b79731389 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:27.004063 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90e1a9-e908-4c00-a580-730b79731389 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:27.005197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90e1a9-e908-4c00-a580-730b79731389 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:27.005724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90e1a9-e908-4c00-a580-730b79731389 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:27.007473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90e1a9-e908-4c00-a580-730b79731389 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:27.033233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 346/681] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Jun 15 15:37:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:27.034971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90e1a9-e908-4c00-a580-730b79731389 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:27.156787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b722d4-8e65-425f-98e5-48881e69604f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:27.174920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:27.175781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90e1a9-e908-4c00-a580-730b79731389 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:27.202562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ae98d-5cea-46a5-ad20-2fd2b1733f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:27.203410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ae98d-5cea-46a5-ad20-2fd2b1733f73 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:27.203695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ae98d-5cea-46a5-ad20-2fd2b1733f73 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:27.203970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ae98d-5cea-46a5-ad20-2fd2b1733f73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:27.207148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:27.211799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ae98d-5cea-46a5-ad20-2fd2b1733f73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:27.213251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ae98d-5cea-46a5-ad20-2fd2b1733f73 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:27.218600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cad1dc-5485-4d43-afe1-cb8d402c585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:27.219553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cad1dc-5485-4d43-afe1-cb8d402c585b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:27.219834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cad1dc-5485-4d43-afe1-cb8d402c585b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:27.220106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cad1dc-5485-4d43-afe1-cb8d402c585b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-2b9ae98d-5cea-46a5-ad20-2fd2b1733f73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:27.229397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:27.230301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:27.231236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:27.231236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:27.231236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:27.231236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:27.231236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:27.231236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:27.231236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:27.231236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:27.231236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 336/682] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:27.238135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ae98d-5cea-46a5-ad20-2fd2b1733f73 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:27.306929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cad1dc-5485-4d43-afe1-cb8d402c585b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:27.308316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cad1dc-5485-4d43-afe1-cb8d402c585b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:27.369042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ae98d-5cea-46a5-ad20-2fd2b1733f73 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:27.408373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4468fc6-8599-412a-bc64-b404b4c9a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:27.409046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4468fc6-8599-412a-bc64-b404b4c9a027 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:27.409492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4468fc6-8599-412a-bc64-b404b4c9a027 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:27.409781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4468fc6-8599-412a-bc64-b404b4c9a027 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:27.410577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4468fc6-8599-412a-bc64-b404b4c9a027 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:27.410924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4468fc6-8599-412a-bc64-b404b4c9a027 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:27.412376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4468fc6-8599-412a-bc64-b404b4c9a027 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:27.427273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 337/683] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:27.438176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4468fc6-8599-412a-bc64-b404b4c9a027 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:27.505048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 347/684] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:27.505407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:27.511367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cad1dc-5485-4d43-afe1-cb8d402c585b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:27.554875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:27.633857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cad1dc-5485-4d43-afe1-cb8d402c585b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:27.671027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4468fc6-8599-412a-bc64-b404b4c9a027 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:27.673521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dbc72-1a20-49f2-8235-4fb3509bc634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:27.674319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dbc72-1a20-49f2-8235-4fb3509bc634 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:27.674601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dbc72-1a20-49f2-8235-4fb3509bc634 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:27.674899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dbc72-1a20-49f2-8235-4fb3509bc634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:27.691990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dbc72-1a20-49f2-8235-4fb3509bc634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:27.693445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dbc72-1a20-49f2-8235-4fb3509bc634 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-585dbc72-1a20-49f2-8235-4fb3509bc634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:27.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:27.724025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:27.724979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:27.724979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:27.724979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:27.724979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:27.724979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 348/685] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:27.729972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dbc72-1a20-49f2-8235-4fb3509bc634 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:27.746599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef415a68-d359-4deb-a477-4cfe627ec9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:27.747300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef415a68-d359-4deb-a477-4cfe627ec9e0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:27.747588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef415a68-d359-4deb-a477-4cfe627ec9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:27.747869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef415a68-d359-4deb-a477-4cfe627ec9e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:27.748843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef415a68-d359-4deb-a477-4cfe627ec9e0 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=bf1915b9919442d692bcedf25f53320c) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:27.749274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef415a68-d359-4deb-a477-4cfe627ec9e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:27.750712 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef415a68-d359-4deb-a477-4cfe627ec9e0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:27.767082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 338/686] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:27 2022] GET /identity/v3/role_assignments?user.id=bf1915b9919442d692bcedf25f53320c&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:27.815559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:27.843701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dbc72-1a20-49f2-8235-4fb3509bc634 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:27.897967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3c5a-b973-4495-b5e0-7895017e8cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:27.898569 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3c5a-b973-4495-b5e0-7895017e8cdd None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:27.898877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3c5a-b973-4495-b5e0-7895017e8cdd None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:27.899156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3c5a-b973-4495-b5e0-7895017e8cdd None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:27.899896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc3c5a-b973-4495-b5e0-7895017e8cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:27.900201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc3c5a-b973-4495-b5e0-7895017e8cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:27.901602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc3c5a-b973-4495-b5e0-7895017e8cdd None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:27.918634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 349/687] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:27.924401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef415a68-d359-4deb-a477-4cfe627ec9e0 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:28.064724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:28.069617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef415a68-d359-4deb-a477-4cfe627ec9e0 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:28.107153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba7b29-12fe-4b1a-8346-7389a904dc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:28.107946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7b29-12fe-4b1a-8346-7389a904dc0f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.108228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7b29-12fe-4b1a-8346-7389a904dc0f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.108504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7b29-12fe-4b1a-8346-7389a904dc0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.116901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba7b29-12fe-4b1a-8346-7389a904dc0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:28.118343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba7b29-12fe-4b1a-8346-7389a904dc0f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-a8ba7b29-12fe-4b1a-8346-7389a904dc0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:28.134203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:28.135143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:28.136040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:28.136040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:28.136040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:28.136040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:28.136040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:28.136040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:28.136040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:28.136040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:28.136040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 339/688] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:28.141288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3c5a-b973-4495-b5e0-7895017e8cdd None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:28.261434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3c5a-b973-4495-b5e0-7895017e8cdd None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:28.302292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69091b-4d3a-48b8-b451-650ea8161d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:28.302741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee390f6-cc34-4696-92ab-a0060dee99f2 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.303607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee390f6-cc34-4696-92ab-a0060dee99f2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.303894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee390f6-cc34-4696-92ab-a0060dee99f2 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.304433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69091b-4d3a-48b8-b451-650ea8161d0d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.304884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69091b-4d3a-48b8-b451-650ea8161d0d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.305355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69091b-4d3a-48b8-b451-650ea8161d0d None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.306335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69091b-4d3a-48b8-b451-650ea8161d0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:28.306440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 340/689] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:28.306952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69091b-4d3a-48b8-b451-650ea8161d0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:28.308565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69091b-4d3a-48b8-b451-650ea8161d0d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:28.310352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5b664-0fc2-42af-8189-2dab7b1f05f0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.310709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:28.310832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5b664-0fc2-42af-8189-2dab7b1f05f0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.310957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5b664-0fc2-42af-8189-2dab7b1f05f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.328579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-29e5b664-0fc2-42af-8189-2dab7b1f05f0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:28.329755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-29e5b664-0fc2-42af-8189-2dab7b1f05f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:28.331355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 350/690] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:28.345446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69091b-4d3a-48b8-b451-650ea8161d0d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:28.424851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 341/691] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:28.474982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf018797-f5e2-4a7e-a4b4-744cb4f97d20 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.475480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf018797-f5e2-4a7e-a4b4-744cb4f97d20 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.475836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf018797-f5e2-4a7e-a4b4-744cb4f97d20 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.477638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 342/692] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:28.479526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf018797-f5e2-4a7e-a4b4-744cb4f97d20 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:28.529512 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69091b-4d3a-48b8-b451-650ea8161d0d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:28.565269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c81d24-1fa8-47fd-b1b4-70977a8ee008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:28.566093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81d24-1fa8-47fd-b1b4-70977a8ee008 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.566379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81d24-1fa8-47fd-b1b4-70977a8ee008 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.566651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81d24-1fa8-47fd-b1b4-70977a8ee008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.567514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c81d24-1fa8-47fd-b1b4-70977a8ee008 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=a6e3bb6553264387936e9a534afc3ac0) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:28.567880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c81d24-1fa8-47fd-b1b4-70977a8ee008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:28.569878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c81d24-1fa8-47fd-b1b4-70977a8ee008 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:28.575366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:28.576345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 351/693] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:28 2022] GET /identity/v3/role_assignments?user.id=a6e3bb6553264387936e9a534afc3ac0&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:28.577831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eced2d-d6de-4a23-af43-efc8fe0ee41c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.578160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eced2d-d6de-4a23-af43-efc8fe0ee41c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.578403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eced2d-d6de-4a23-af43-efc8fe0ee41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.597060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e2eced2d-d6de-4a23-af43-efc8fe0ee41c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:28.598320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e2eced2d-d6de-4a23-af43-efc8fe0ee41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:28.608773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf018797-f5e2-4a7e-a4b4-744cb4f97d20 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:28.659707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:28.666570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65beb9-94f8-4db8-8d10-4b388dd3df0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:28.667428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65beb9-94f8-4db8-8d10-4b388dd3df0a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.667713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65beb9-94f8-4db8-8d10-4b388dd3df0a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.667990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65beb9-94f8-4db8-8d10-4b388dd3df0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.677005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 352/694] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:28.678698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65beb9-94f8-4db8-8d10-4b388dd3df0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-9c65beb9-94f8-4db8-8d10-4b388dd3df0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:28.686840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:28.687840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:28.688683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:28.692761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 343/695] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:28.694923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eced2d-d6de-4a23-af43-efc8fe0ee41c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:28.724534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65beb9-94f8-4db8-8d10-4b388dd3df0a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:28.815797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eced2d-d6de-4a23-af43-efc8fe0ee41c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:28.854795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ab359-aa02-492c-891c-4f9a80c9c0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:28.855912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ab359-aa02-492c-891c-4f9a80c9c0d2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.856390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ab359-aa02-492c-891c-4f9a80c9c0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.856879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ab359-aa02-492c-891c-4f9a80c9c0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.857910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ab359-aa02-492c-891c-4f9a80c9c0d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:28.858520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ab359-aa02-492c-891c-4f9a80c9c0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:28.860112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ab359-aa02-492c-891c-4f9a80c9c0d2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:28.864341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65beb9-94f8-4db8-8d10-4b388dd3df0a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:28.869859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 353/696] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:28.876262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ab359-aa02-492c-891c-4f9a80c9c0d2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:28.905882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54941c72-4148-46a8-abdf-17b818e0e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:28.906683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54941c72-4148-46a8-abdf-17b818e0e44f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:28.906995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54941c72-4148-46a8-abdf-17b818e0e44f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:28.907278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54941c72-4148-46a8-abdf-17b818e0e44f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:28.918871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54941c72-4148-46a8-abdf-17b818e0e44f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:28.920314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54941c72-4148-46a8-abdf-17b818e0e44f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:28.928713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:28.933970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-54941c72-4148-46a8-abdf-17b818e0e44f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:28.939690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:28.940633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:28.941516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:28.941516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:28.941516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:28.941516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:28.941516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 344/697] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:28.950780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54941c72-4148-46a8-abdf-17b818e0e44f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:29.000383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ab359-aa02-492c-891c-4f9a80c9c0d2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:29.038287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a938623-df2d-46ce-b8cb-a551e1696f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:29.039122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a938623-df2d-46ce-b8cb-a551e1696f88 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.039500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a938623-df2d-46ce-b8cb-a551e1696f88 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.039782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a938623-df2d-46ce-b8cb-a551e1696f88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.051797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a938623-df2d-46ce-b8cb-a551e1696f88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:29.054025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a938623-df2d-46ce-b8cb-a551e1696f88 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:29.059032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54941c72-4148-46a8-abdf-17b818e0e44f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-2a938623-df2d-46ce-b8cb-a551e1696f88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:29.073711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:29.074647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:29.075572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:29.075572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:29.075572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:29.075572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:29.076263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 354/698] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:29.081254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a938623-df2d-46ce-b8cb-a551e1696f88 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:29.097294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937672ad-8fb2-4a4c-8bd0-a8553ad564a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:29.097965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937672ad-8fb2-4a4c-8bd0-a8553ad564a5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.098250 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937672ad-8fb2-4a4c-8bd0-a8553ad564a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.098525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937672ad-8fb2-4a4c-8bd0-a8553ad564a5 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.099278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937672ad-8fb2-4a4c-8bd0-a8553ad564a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:29.099588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937672ad-8fb2-4a4c-8bd0-a8553ad564a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:29.101020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937672ad-8fb2-4a4c-8bd0-a8553ad564a5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:29.121027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 345/699] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:29.126262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937672ad-8fb2-4a4c-8bd0-a8553ad564a5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:29.200008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:29.216984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a938623-df2d-46ce-b8cb-a551e1696f88 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:29.220091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:29.251689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937672ad-8fb2-4a4c-8bd0-a8553ad564a5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:29.267552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfabffa-a1ea-458a-bb38-d05d1d30ebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:29.268225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfabffa-a1ea-458a-bb38-d05d1d30ebfa None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.268513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfabffa-a1ea-458a-bb38-d05d1d30ebfa None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.268793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfabffa-a1ea-458a-bb38-d05d1d30ebfa None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.269516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfabffa-a1ea-458a-bb38-d05d1d30ebfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:29.269823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfabffa-a1ea-458a-bb38-d05d1d30ebfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:29.271505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfabffa-a1ea-458a-bb38-d05d1d30ebfa None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:29.284845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 355/700] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:29.290914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfabffa-a1ea-458a-bb38-d05d1d30ebfa None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:29.301991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514af7da-040f-4a34-9aec-b08b90995e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:29.303570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af7da-040f-4a34-9aec-b08b90995e2f None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.304467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af7da-040f-4a34-9aec-b08b90995e2f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.304952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af7da-040f-4a34-9aec-b08b90995e2f None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.305872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514af7da-040f-4a34-9aec-b08b90995e2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:29.307527 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514af7da-040f-4a34-9aec-b08b90995e2f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:29.333799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 346/701] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:37:29 2022] POST /identity/v3/users => generated 254 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:37:29.495261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfabffa-a1ea-458a-bb38-d05d1d30ebfa None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:29.552590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c659b-23f9-4404-8ae1-fcb907a04b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:29.553948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c659b-23f9-4404-8ae1-fcb907a04b6a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.554229 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c659b-23f9-4404-8ae1-fcb907a04b6a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.554505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c659b-23f9-4404-8ae1-fcb907a04b6a None admin] PATH_INFO: `/v3/users/nova` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.570180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c659b-23f9-4404-8ae1-fcb907a04b6a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:29.571721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c659b-23f9-4404-8ae1-fcb907a04b6a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-6f5c659b-23f9-4404-8ae1-fcb907a04b6a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:29.591073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:29.592072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:29.593651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 356/702] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Jun 15 15:37:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:29.598904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514af7da-040f-4a34-9aec-b08b90995e2f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:29.643973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ea07c-b024-4c5f-b9a3-6a9bb14871c5 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.644296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ea07c-b024-4c5f-b9a3-6a9bb14871c5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.644527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ea07c-b024-4c5f-b9a3-6a9bb14871c5 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.646148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 357/703] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:29.651990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c2ce4-9118-4d0a-b409-d9e773574bf8 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.652271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c2ce4-9118-4d0a-b409-d9e773574bf8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.652540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c2ce4-9118-4d0a-b409-d9e773574bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.673524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-895c2ce4-9118-4d0a-b409-d9e773574bf8 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:29.674711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-895c2ce4-9118-4d0a-b409-d9e773574bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:29.694128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:29.722555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514af7da-040f-4a34-9aec-b08b90995e2f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:29.744380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:29.751279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 358/704] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:29.764477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22b02d-b975-4a98-9e82-26bf7a633139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:29.765580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22b02d-b975-4a98-9e82-26bf7a633139 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.765973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22b02d-b975-4a98-9e82-26bf7a633139 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.766355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22b02d-b975-4a98-9e82-26bf7a633139 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.767359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22b02d-b975-4a98-9e82-26bf7a633139 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:29.767776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22b02d-b975-4a98-9e82-26bf7a633139 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:29.769613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22b02d-b975-4a98-9e82-26bf7a633139 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:29.778145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c2ce4-9118-4d0a-b409-d9e773574bf8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:29.797871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 347/705] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Jun 15 15:37:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:29.799564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22b02d-b975-4a98-9e82-26bf7a633139 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:29.914994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c2ce4-9118-4d0a-b409-d9e773574bf8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:29.925093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22b02d-b975-4a98-9e82-26bf7a633139 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:29.968235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725b48b-f869-4d51-80d7-aca0120030f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:29.969789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725b48b-f869-4d51-80d7-aca0120030f2 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.970297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725b48b-f869-4d51-80d7-aca0120030f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.970753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725b48b-f869-4d51-80d7-aca0120030f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.975942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59d935-827b-41ba-9c7e-52a544e94a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:29.976903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59d935-827b-41ba-9c7e-52a544e94a47 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:29.982755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59d935-827b-41ba-9c7e-52a544e94a47 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:29.982755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59d935-827b-41ba-9c7e-52a544e94a47 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:29.982755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59d935-827b-41ba-9c7e-52a544e94a47 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:29.982755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59d935-827b-41ba-9c7e-52a544e94a47 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-9a59d935-827b-41ba-9c7e-52a544e94a47 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:30.008285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:30.009334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 15:37:30.010180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:30.015924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 359/706] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Jun 15 15:37:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:30.019373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:30.022044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59d935-827b-41ba-9c7e-52a544e94a47 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:30.047136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725b48b-f869-4d51-80d7-aca0120030f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:30.048585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725b48b-f869-4d51-80d7-aca0120030f2 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:30.061619 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:30.220445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59d935-827b-41ba-9c7e-52a544e94a47 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:30.231183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 348/707] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:30.239668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725b48b-f869-4d51-80d7-aca0120030f2 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:30.267885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c611fd-3b37-4930-b578-910708df03ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:30.268554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c611fd-3b37-4930-b578-910708df03ae None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:30.268830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c611fd-3b37-4930-b578-910708df03ae None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:30.269099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c611fd-3b37-4930-b578-910708df03ae None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:30.269804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c611fd-3b37-4930-b578-910708df03ae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:30.270358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c611fd-3b37-4930-b578-910708df03ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:30.272295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c611fd-3b37-4930-b578-910708df03ae None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:30.283820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 360/708] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Jun 15 15:37:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:30.289297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c611fd-3b37-4930-b578-910708df03ae None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:30.367094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:30.419516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725b48b-f869-4d51-80d7-aca0120030f2 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:30.427058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:30.462813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c611fd-3b37-4930-b578-910708df03ae None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:30.502143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8e5b3-5cef-43de-a608-1eaf9ce1ff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:30.502920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8e5b3-5cef-43de-a608-1eaf9ce1ff02 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:30.503203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8e5b3-5cef-43de-a608-1eaf9ce1ff02 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:30.503474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8e5b3-5cef-43de-a608-1eaf9ce1ff02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:30.525485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8e5b3-5cef-43de-a608-1eaf9ce1ff02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:30.526961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8e5b3-5cef-43de-a608-1eaf9ce1ff02 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:30.540673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd10e1-d427-43a3-965c-5fc4d4e429f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:30.541707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd10e1-d427-43a3-965c-5fc4d4e429f0 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:30.542202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd10e1-d427-43a3-965c-5fc4d4e429f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:30.542706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd10e1-d427-43a3-965c-5fc4d4e429f0 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:30.543626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd10e1-d427-43a3-965c-5fc4d4e429f0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:30.544142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd10e1-d427-43a3-965c-5fc4d4e429f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:30.545707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd10e1-d427-43a3-965c-5fc4d4e429f0 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-84b8e5b3-5cef-43de-a608-1eaf9ce1ff02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:30.551581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:30.552544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:30.554054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:30.554054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:30.554054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:30.554054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:30.554054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 349/709] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:30.555386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 361/710] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Jun 15 15:37:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:30.566773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489072a-8aea-4053-b869-9f3079bce75e None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:30.566773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489072a-8aea-4053-b869-9f3079bce75e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:30.566773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489072a-8aea-4053-b869-9f3079bce75e None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:30.566773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 350/711] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:30.567199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd10e1-d427-43a3-965c-5fc4d4e429f0 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:30.573611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b57b3-b628-4b44-8ee0-1e956c1c82e1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:30.573955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b57b3-b628-4b44-8ee0-1e956c1c82e1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:30.574157 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b57b3-b628-4b44-8ee0-1e956c1c82e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:30.606180 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-fa4b57b3-b628-4b44-8ee0-1e956c1c82e1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:30.607444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fa4b57b3-b628-4b44-8ee0-1e956c1c82e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:30.719498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 351/712] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:30.721257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd10e1-d427-43a3-965c-5fc4d4e429f0 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:30.742416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b57b3-b628-4b44-8ee0-1e956c1c82e1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:30.781633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46e040-d4fd-46ff-96d0-a8008eaef7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:30.782639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46e040-d4fd-46ff-96d0-a8008eaef7a2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:30.783160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46e040-d4fd-46ff-96d0-a8008eaef7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:30.783643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46e040-d4fd-46ff-96d0-a8008eaef7a2 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:30.784572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46e040-d4fd-46ff-96d0-a8008eaef7a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:30.785114 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46e040-d4fd-46ff-96d0-a8008eaef7a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:30.786693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46e040-d4fd-46ff-96d0-a8008eaef7a2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:30.800805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 362/713] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:30.806855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46e040-d4fd-46ff-96d0-a8008eaef7a2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:30.811229 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:30.819284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:30.907625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b57b3-b628-4b44-8ee0-1e956c1c82e1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:30.960158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46e040-d4fd-46ff-96d0-a8008eaef7a2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:30.981344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30960870-4f5d-4e7b-9937-4517d60b4d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:30.982790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30960870-4f5d-4e7b-9937-4517d60b4d85 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:30.983229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30960870-4f5d-4e7b-9937-4517d60b4d85 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:30.983695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30960870-4f5d-4e7b-9937-4517d60b4d85 None admin] PATH_INFO: `/v3/services/network` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:30.986821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30960870-4f5d-4e7b-9937-4517d60b4d85 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:30.988530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30960870-4f5d-4e7b-9937-4517d60b4d85 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-30960870-4f5d-4e7b-9937-4517d60b4d85 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:31.017400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:31.018662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 15:37:31.019734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:31.034840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 352/714] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:30 2022] GET /identity/v3/services/network => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:31.035976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30960870-4f5d-4e7b-9937-4517d60b4d85 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:31.053436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40fe82-f522-447f-8afa-ee10b47bcc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:31.054515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40fe82-f522-447f-8afa-ee10b47bcc3c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:31.055075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40fe82-f522-447f-8afa-ee10b47bcc3c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:31.055500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40fe82-f522-447f-8afa-ee10b47bcc3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:31.069366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40fe82-f522-447f-8afa-ee10b47bcc3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:31.071096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40fe82-f522-447f-8afa-ee10b47bcc3c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-7a40fe82-f522-447f-8afa-ee10b47bcc3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:31.088565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:31.089651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:31.090700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:31.090700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:31.090700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:31.090700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:31.090700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:31.090700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:31.090700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:31.090700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:31.093399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 363/715] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:31.098923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40fe82-f522-447f-8afa-ee10b47bcc3c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:31.183134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30960870-4f5d-4e7b-9937-4517d60b4d85 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:31.220081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc8c9c-002a-463a-ac78-a70d247eed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:31.221206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc8c9c-002a-463a-ac78-a70d247eed7d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:31.221645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc8c9c-002a-463a-ac78-a70d247eed7d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:31.222075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc8c9c-002a-463a-ac78-a70d247eed7d None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:31.222977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc8c9c-002a-463a-ac78-a70d247eed7d None admin] RBAC: Adding query filter params (name=network) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:31.223437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc8c9c-002a-463a-ac78-a70d247eed7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:31.225013 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc8c9c-002a-463a-ac78-a70d247eed7d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:31.235100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:31.235957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 353/716] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:37:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:31.237007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc8c9c-002a-463a-ac78-a70d247eed7d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:31.308084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:31.354511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc8c9c-002a-463a-ac78-a70d247eed7d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:31.391590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40fe82-f522-447f-8afa-ee10b47bcc3c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:31.422139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b0df2-f6c8-4e42-91ea-6d4d6db296ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:31.422827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b0df2-f6c8-4e42-91ea-6d4d6db296ea None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:31.423108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b0df2-f6c8-4e42-91ea-6d4d6db296ea None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:31.423382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b0df2-f6c8-4e42-91ea-6d4d6db296ea None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:31.424083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b0df2-f6c8-4e42-91ea-6d4d6db296ea None admin] RBAC: Adding query filter params (type=network) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:31.424390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b0df2-f6c8-4e42-91ea-6d4d6db296ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:31.425747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b0df2-f6c8-4e42-91ea-6d4d6db296ea None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:31.434957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 354/717] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:37:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:31.472455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8259a-5f99-42df-837b-86aee7b541e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:31.473533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8259a-5f99-42df-837b-86aee7b541e8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:31.474032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8259a-5f99-42df-837b-86aee7b541e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:31.474513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8259a-5f99-42df-837b-86aee7b541e8 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:31.476134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8259a-5f99-42df-837b-86aee7b541e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:31.476676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8259a-5f99-42df-837b-86aee7b541e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:31.478528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8259a-5f99-42df-837b-86aee7b541e8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:31.497511 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 364/718] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:31.503988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b0df2-f6c8-4e42-91ea-6d4d6db296ea None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:31.610215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310832c-7a32-4330-97ed-ef562a63514a None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:31.610515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310832c-7a32-4330-97ed-ef562a63514a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:31.610698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310832c-7a32-4330-97ed-ef562a63514a None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:31.612572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 365/719] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:31.618257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b00d55-6a46-478c-9f34-f1a1f4854c48 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:31.618563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b00d55-6a46-478c-9f34-f1a1f4854c48 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:31.618846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b00d55-6a46-478c-9f34-f1a1f4854c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:31.625909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:31.639195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-41b00d55-6a46-478c-9f34-f1a1f4854c48 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:31.640653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-41b00d55-6a46-478c-9f34-f1a1f4854c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:31.704809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b0df2-f6c8-4e42-91ea-6d4d6db296ea None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:31.741562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 366/720] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:31.745789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dd2db-5ea5-46ff-a54d-21449e4008fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:31.746678 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dd2db-5ea5-46ff-a54d-21449e4008fb None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:31.747224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dd2db-5ea5-46ff-a54d-21449e4008fb None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:31.747708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dd2db-5ea5-46ff-a54d-21449e4008fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:31.748735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dd2db-5ea5-46ff-a54d-21449e4008fb None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=4327e4eed5a1436ea1a00c2209bc5750) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:31.749267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dd2db-5ea5-46ff-a54d-21449e4008fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:31.750924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dd2db-5ea5-46ff-a54d-21449e4008fb None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:31.757863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 355/721] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:31 2022] GET /identity/v3/role_assignments?user.id=4327e4eed5a1436ea1a00c2209bc5750&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=433478cc318a467c931f13a757f161fa => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:31.793410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b00d55-6a46-478c-9f34-f1a1f4854c48 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:31.869233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:31.922338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b00d55-6a46-478c-9f34-f1a1f4854c48 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:31.963057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83452159-9086-4478-b3d9-7cae3482b9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:31.963980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83452159-9086-4478-b3d9-7cae3482b9c8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:31.964269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83452159-9086-4478-b3d9-7cae3482b9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:31.964549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83452159-9086-4478-b3d9-7cae3482b9c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:31.978583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83452159-9086-4478-b3d9-7cae3482b9c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-83452159-9086-4478-b3d9-7cae3482b9c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:31.986526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:31.987484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:31.988528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 367/722] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:31.993599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dd2db-5ea5-46ff-a54d-21449e4008fb None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:32.101116 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dd2db-5ea5-46ff-a54d-21449e4008fb None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:32.122230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:32.148891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02da378-e3ea-4aa7-9296-fe86fd98ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:32.149934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02da378-e3ea-4aa7-9296-fe86fd98ab77 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:32.150416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02da378-e3ea-4aa7-9296-fe86fd98ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:32.150932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02da378-e3ea-4aa7-9296-fe86fd98ab77 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:32.151875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02da378-e3ea-4aa7-9296-fe86fd98ab77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:32.152378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02da378-e3ea-4aa7-9296-fe86fd98ab77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:32.153936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02da378-e3ea-4aa7-9296-fe86fd98ab77 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:32.163087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 356/723] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:32.175219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83452159-9086-4478-b3d9-7cae3482b9c8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:32.276444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd76c5-1821-4479-9529-bfeb467fe1a8 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:32.277100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd76c5-1821-4479-9529-bfeb467fe1a8 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:32.277576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd76c5-1821-4479-9529-bfeb467fe1a8 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:32.280504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 357/724] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:32.284734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735459b-80a9-4d11-852d-9e145d77c421 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:32.285251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735459b-80a9-4d11-852d-9e145d77c421 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:32.285748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735459b-80a9-4d11-852d-9e145d77c421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:32.304407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8735459b-80a9-4d11-852d-9e145d77c421 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:32.305888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8735459b-80a9-4d11-852d-9e145d77c421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:32.367122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83452159-9086-4478-b3d9-7cae3482b9c8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:32.382972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 358/725] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 15:37:32.393492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:32.404919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735459b-80a9-4d11-852d-9e145d77c421 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:32.439615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce59e0-2c31-4ecc-9b1f-5a4a51573dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:32.440428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce59e0-2c31-4ecc-9b1f-5a4a51573dd1 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:32.440713 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce59e0-2c31-4ecc-9b1f-5a4a51573dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:32.440992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce59e0-2c31-4ecc-9b1f-5a4a51573dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:32.470814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce59e0-2c31-4ecc-9b1f-5a4a51573dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:32.472346 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce59e0-2c31-4ecc-9b1f-5a4a51573dd1 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-56ce59e0-2c31-4ecc-9b1f-5a4a51573dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:32.497009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:32.498095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:32.499777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:32.499777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:32.499777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:32.499777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:32.499777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 368/726] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:32.503661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce59e0-2c31-4ecc-9b1f-5a4a51573dd1 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:32.521433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735459b-80a9-4d11-852d-9e145d77c421 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:32.524644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:32.561893 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae0fa5-d287-4a86-8d10-b40bca8832e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:32.563093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae0fa5-d287-4a86-8d10-b40bca8832e9 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:32.563604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae0fa5-d287-4a86-8d10-b40bca8832e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:32.564122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae0fa5-d287-4a86-8d10-b40bca8832e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:32.575363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae0fa5-d287-4a86-8d10-b40bca8832e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-86ae0fa5-d287-4a86-8d10-b40bca8832e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:32.583425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:32.584927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:32.585766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:32.593165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 359/727] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:32.594652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae0fa5-d287-4a86-8d10-b40bca8832e9 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:32.701131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae0fa5-d287-4a86-8d10-b40bca8832e9 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:32.735232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c2bd7-1061-4fb0-b7aa-01d59d56755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:32.736257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2bd7-1061-4fb0-b7aa-01d59d56755c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:32.736745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2bd7-1061-4fb0-b7aa-01d59d56755c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:32.737221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2bd7-1061-4fb0-b7aa-01d59d56755c None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:32.738188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c2bd7-1061-4fb0-b7aa-01d59d56755c None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:32.738696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c2bd7-1061-4fb0-b7aa-01d59d56755c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:32.740922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c2bd7-1061-4fb0-b7aa-01d59d56755c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:32.753992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 360/728] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:32.755528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c2bd7-1061-4fb0-b7aa-01d59d56755c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:32.768879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce59e0-2c31-4ecc-9b1f-5a4a51573dd1 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:32.799104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:32.859530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c2bd7-1061-4fb0-b7aa-01d59d56755c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:32.884435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355e394-2bd2-4211-a6a8-5d9b377cee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:32.885127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355e394-2bd2-4211-a6a8-5d9b377cee52 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:32.885409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355e394-2bd2-4211-a6a8-5d9b377cee52 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:32.885688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355e394-2bd2-4211-a6a8-5d9b377cee52 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:32.886531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355e394-2bd2-4211-a6a8-5d9b377cee52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:32.886818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355e394-2bd2-4211-a6a8-5d9b377cee52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:32.888213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355e394-2bd2-4211-a6a8-5d9b377cee52 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:32.899074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0da173-ed68-4a0a-8110-d7e87cab2c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:32.900290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da173-ed68-4a0a-8110-d7e87cab2c06 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:32.900780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da173-ed68-4a0a-8110-d7e87cab2c06 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:32.901254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da173-ed68-4a0a-8110-d7e87cab2c06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:32.913269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0da173-ed68-4a0a-8110-d7e87cab2c06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:32.915060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0da173-ed68-4a0a-8110-d7e87cab2c06 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:32.921304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 369/729] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:32.926241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355e394-2bd2-4211-a6a8-5d9b377cee52 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-1c0da173-ed68-4a0a-8110-d7e87cab2c06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:32.934548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:32.935459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:32.936332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:32.936332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:32.936332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:32.936332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:32.943721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 361/730] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:32.945178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0da173-ed68-4a0a-8110-d7e87cab2c06 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:32.966710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:33.036094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:33.059680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0da173-ed68-4a0a-8110-d7e87cab2c06 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:33.096149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d1bdc-cd0b-4635-91b0-b38b3f011761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:33.097117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355e394-2bd2-4211-a6a8-5d9b377cee52 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:33.097421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d1bdc-cd0b-4635-91b0-b38b3f011761 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.097908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d1bdc-cd0b-4635-91b0-b38b3f011761 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.098381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d1bdc-cd0b-4635-91b0-b38b3f011761 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.099411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d1bdc-cd0b-4635-91b0-b38b3f011761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:33.100456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d1bdc-cd0b-4635-91b0-b38b3f011761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:33.102071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d1bdc-cd0b-4635-91b0-b38b3f011761 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:33.119953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 362/731] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:33.121516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d1bdc-cd0b-4635-91b0-b38b3f011761 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:33.162938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f418f83-794e-401f-9661-445f6a982c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:33.163745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418f83-794e-401f-9661-445f6a982c5b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.164029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418f83-794e-401f-9661-445f6a982c5b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.164304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418f83-794e-401f-9661-445f6a982c5b None admin] PATH_INFO: `/v3/users/swift` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.176255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f418f83-794e-401f-9661-445f6a982c5b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:33.178371 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f418f83-794e-401f-9661-445f6a982c5b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-6f418f83-794e-401f-9661-445f6a982c5b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:33.210683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:33.211696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:33.212633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:33.216451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 370/732] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:33.218036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6de3c-7aa6-4c49-bd55-bd9cdb5ba110 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.218273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6de3c-7aa6-4c49-bd55-bd9cdb5ba110 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.218607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6de3c-7aa6-4c49-bd55-bd9cdb5ba110 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.220140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 371/733] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:33.220842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6de3c-7aa6-4c49-bd55-bd9cdb5ba110 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:33.234754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d1bdc-cd0b-4635-91b0-b38b3f011761 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:33.271671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bec7b2-344d-47be-8133-279ea4eabc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:33.272918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bec7b2-344d-47be-8133-279ea4eabc6a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.273428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bec7b2-344d-47be-8133-279ea4eabc6a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.273914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bec7b2-344d-47be-8133-279ea4eabc6a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.281695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bec7b2-344d-47be-8133-279ea4eabc6a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:33.283489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bec7b2-344d-47be-8133-279ea4eabc6a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:33.285533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-76bec7b2-344d-47be-8133-279ea4eabc6a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:33.301030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:33.301992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:33.303602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:33.311449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 363/734] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:37:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:33.313064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d784c-2086-4d5d-b1e8-8a1119bc07f1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.313635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d784c-2086-4d5d-b1e8-8a1119bc07f1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.314202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d784c-2086-4d5d-b1e8-8a1119bc07f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.317985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:33.334961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-482d784c-2086-4d5d-b1e8-8a1119bc07f1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:33.335313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6de3c-7aa6-4c49-bd55-bd9cdb5ba110 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:33.336711 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-482d784c-2086-4d5d-b1e8-8a1119bc07f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:33.374592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf65ff8-c720-4ef8-893c-09d0485bfa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:33.375293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf65ff8-c720-4ef8-893c-09d0485bfa2f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.375547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf65ff8-c720-4ef8-893c-09d0485bfa2f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.375823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf65ff8-c720-4ef8-893c-09d0485bfa2f None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.377382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf65ff8-c720-4ef8-893c-09d0485bfa2f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:33.377382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf65ff8-c720-4ef8-893c-09d0485bfa2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:33.378508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf65ff8-c720-4ef8-893c-09d0485bfa2f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:33.401927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 372/735] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Jun 15 15:37:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:33.402903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf65ff8-c720-4ef8-893c-09d0485bfa2f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:33.411727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 364/736] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:33.413509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d784c-2086-4d5d-b1e8-8a1119bc07f1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:33.522859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d784c-2086-4d5d-b1e8-8a1119bc07f1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:33.530358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf65ff8-c720-4ef8-893c-09d0485bfa2f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:33.559200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:33.563401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba720b-5930-4b23-bda9-065bc9e2d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:33.564480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba720b-5930-4b23-bda9-065bc9e2d4fe None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.565364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba720b-5930-4b23-bda9-065bc9e2d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.565878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba720b-5930-4b23-bda9-065bc9e2d4fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.566103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad9f45-11ba-465a-b8bc-3e318103470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:33.566798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad9f45-11ba-465a-b8bc-3e318103470b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.567085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad9f45-11ba-465a-b8bc-3e318103470b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.567357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad9f45-11ba-465a-b8bc-3e318103470b None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.568200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad9f45-11ba-465a-b8bc-3e318103470b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:33.568515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad9f45-11ba-465a-b8bc-3e318103470b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:33.570045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad9f45-11ba-465a-b8bc-3e318103470b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:33.592611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 373/737] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Jun 15 15:37:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:33.592611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad9f45-11ba-465a-b8bc-3e318103470b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:33.594328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:33.647196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba720b-5930-4b23-bda9-065bc9e2d4fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:33.648656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba720b-5930-4b23-bda9-065bc9e2d4fe None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:33.722205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad9f45-11ba-465a-b8bc-3e318103470b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:33.759899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f2cb2-aefb-44ee-9ae4-f3522ab1d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:33.760576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f2cb2-aefb-44ee-9ae4-f3522ab1d6d0 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.760859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f2cb2-aefb-44ee-9ae4-f3522ab1d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.761137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f2cb2-aefb-44ee-9ae4-f3522ab1d6d0 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.761873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f2cb2-aefb-44ee-9ae4-f3522ab1d6d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:33.762111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 365/738] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:33.763156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba720b-5930-4b23-bda9-065bc9e2d4fe None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:33.767909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f2cb2-aefb-44ee-9ae4-f3522ab1d6d0 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:33.795309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 374/739] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:37:33 2022] POST /identity/v3/services => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:33.800476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:33.800674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f2cb2-aefb-44ee-9ae4-f3522ab1d6d0 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:33.870247 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:33.927245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f2cb2-aefb-44ee-9ae4-f3522ab1d6d0 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:33.957618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba720b-5930-4b23-bda9-065bc9e2d4fe None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:33.972004 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7e04a-cb84-42b5-bdec-54553ab3f96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:33.972832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7e04a-cb84-42b5-bdec-54553ab3f96c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:33.973117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7e04a-cb84-42b5-bdec-54553ab3f96c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:33.973396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7e04a-cb84-42b5-bdec-54553ab3f96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:33.986921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7e04a-cb84-42b5-bdec-54553ab3f96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:33.988514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7e04a-cb84-42b5-bdec-54553ab3f96c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:33.999805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a6e0e-e4be-4889-b461-d00a5aa1a6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:34.000784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6e0e-e4be-4889-b461-d00a5aa1a6cc None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.001244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6e0e-e4be-4889-b461-d00a5aa1a6cc None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.001690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6e0e-e4be-4889-b461-d00a5aa1a6cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-a1e7e04a-cb84-42b5-bdec-54553ab3f96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:34.008909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:34.009865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:34.011034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:34.011034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:34.011034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:34.011034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:34.011034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 375/740] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:34.016403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea64f4b-c445-48a9-8f8a-4a088ee87009 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.016694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea64f4b-c445-48a9-8f8a-4a088ee87009 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.016975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea64f4b-c445-48a9-8f8a-4a088ee87009 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.018854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 376/741] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:34.020733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea64f4b-c445-48a9-8f8a-4a088ee87009 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:34.080177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a6e0e-e4be-4889-b461-d00a5aa1a6cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:34.082441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a6e0e-e4be-4889-b461-d00a5aa1a6cc None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:34.112569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:34.152966 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea64f4b-c445-48a9-8f8a-4a088ee87009 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:34.167974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:34.194972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0d622-529b-4071-b27f-6519b2aacf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:34.195743 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0d622-529b-4071-b27f-6519b2aacf7e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.196020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0d622-529b-4071-b27f-6519b2aacf7e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.196302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0d622-529b-4071-b27f-6519b2aacf7e None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.197061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0d622-529b-4071-b27f-6519b2aacf7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:34.197328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0d622-529b-4071-b27f-6519b2aacf7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:34.198837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0d622-529b-4071-b27f-6519b2aacf7e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:34.207041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 366/742] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:33 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 15:37:34.208580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cb173-dd68-41df-8ded-4f0797248ccf None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.209677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cb173-dd68-41df-8ded-4f0797248ccf None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.209961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cb173-dd68-41df-8ded-4f0797248ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.213614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 377/743] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:34.214679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0d622-529b-4071-b27f-6519b2aacf7e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:34.232226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c97cb173-dd68-41df-8ded-4f0797248ccf None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:34.233456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c97cb173-dd68-41df-8ded-4f0797248ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:34.306767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 367/744] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 15:37:34.307634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cb173-dd68-41df-8ded-4f0797248ccf None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:34.324384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0d622-529b-4071-b27f-6519b2aacf7e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:34.362123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972891a6-99b0-4d40-aad9-53b7a598383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:34.362858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972891a6-99b0-4d40-aad9-53b7a598383f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.363138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972891a6-99b0-4d40-aad9-53b7a598383f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.363406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972891a6-99b0-4d40-aad9-53b7a598383f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.370314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:34.376437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972891a6-99b0-4d40-aad9-53b7a598383f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:34.377890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972891a6-99b0-4d40-aad9-53b7a598383f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-972891a6-99b0-4d40-aad9-53b7a598383f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:34.397706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:34.398684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:34.399612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:34.399612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:34.399612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:34.399612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:34.402909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 378/745] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:34.404016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972891a6-99b0-4d40-aad9-53b7a598383f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:34.419652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cb173-dd68-41df-8ded-4f0797248ccf None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:34.433204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:34.478551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd35ea-962e-47ea-96c8-d5eee19920d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:34.480622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd35ea-962e-47ea-96c8-d5eee19920d9 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.481066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd35ea-962e-47ea-96c8-d5eee19920d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.481348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd35ea-962e-47ea-96c8-d5eee19920d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.491766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd35ea-962e-47ea-96c8-d5eee19920d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:34.493226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd35ea-962e-47ea-96c8-d5eee19920d9 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-85fd35ea-962e-47ea-96c8-d5eee19920d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:34.507852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:34.508918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 368/746] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:34.509925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd35ea-962e-47ea-96c8-d5eee19920d9 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:34.537043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972891a6-99b0-4d40-aad9-53b7a598383f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:34.575230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b31f7-5746-47d4-a8f7-64ffa49cc907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:34.576169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b31f7-5746-47d4-a8f7-64ffa49cc907 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.576604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b31f7-5746-47d4-a8f7-64ffa49cc907 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.577139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b31f7-5746-47d4-a8f7-64ffa49cc907 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.578124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b31f7-5746-47d4-a8f7-64ffa49cc907 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:34.579673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b31f7-5746-47d4-a8f7-64ffa49cc907 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:34.598253 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 379/747] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:37:34 2022] POST /identity/v3/services => generated 240 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:34.599553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95444c-00e2-4bb5-8209-b63e043f7d53 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.599842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95444c-00e2-4bb5-8209-b63e043f7d53 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.600121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95444c-00e2-4bb5-8209-b63e043f7d53 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.601533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 380/748] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:34.603597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95444c-00e2-4bb5-8209-b63e043f7d53 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:34.635456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd35ea-962e-47ea-96c8-d5eee19920d9 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:34.657140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:34.691597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8bc86-cdab-4250-a77e-400e88ac4d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:34.692856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8bc86-cdab-4250-a77e-400e88ac4d67 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.693938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8bc86-cdab-4250-a77e-400e88ac4d67 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.694463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8bc86-cdab-4250-a77e-400e88ac4d67 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.695541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8bc86-cdab-4250-a77e-400e88ac4d67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:34.696105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8bc86-cdab-4250-a77e-400e88ac4d67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:34.697744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8bc86-cdab-4250-a77e-400e88ac4d67 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:34.711988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 369/749] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:34.713723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7c74c-2da0-4dcc-ae36-28046f75c226 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.714222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7c74c-2da0-4dcc-ae36-28046f75c226 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.714703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7c74c-2da0-4dcc-ae36-28046f75c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.733255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0ba7c74c-2da0-4dcc-ae36-28046f75c226 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:34.734885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0ba7c74c-2da0-4dcc-ae36-28046f75c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:34.739172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:34.750510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95444c-00e2-4bb5-8209-b63e043f7d53 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:34.798326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279e79e-e58f-4deb-89b9-fc3469f945a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:34.799056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279e79e-e58f-4deb-89b9-fc3469f945a1 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.800478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279e79e-e58f-4deb-89b9-fc3469f945a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.800478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279e79e-e58f-4deb-89b9-fc3469f945a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.801067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279e79e-e58f-4deb-89b9-fc3469f945a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:34.801387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279e79e-e58f-4deb-89b9-fc3469f945a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:34.803540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279e79e-e58f-4deb-89b9-fc3469f945a1 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:34.817122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 381/750] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:34.817927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279e79e-e58f-4deb-89b9-fc3469f945a1 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:34.828104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 370/751] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:34.829318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7c74c-2da0-4dcc-ae36-28046f75c226 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:34.946283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7c74c-2da0-4dcc-ae36-28046f75c226 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:34.949498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:34.953632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279e79e-e58f-4deb-89b9-fc3469f945a1 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:34.984796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc027491-3769-4ff1-861b-22878f4b0562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:34.985545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc027491-3769-4ff1-861b-22878f4b0562 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.985906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc027491-3769-4ff1-861b-22878f4b0562 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.986279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc027491-3769-4ff1-861b-22878f4b0562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:34.987383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc027491-3769-4ff1-861b-22878f4b0562 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=a6e3bb6553264387936e9a534afc3ac0) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:34.987705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc027491-3769-4ff1-861b-22878f4b0562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:34.989135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc027491-3769-4ff1-861b-22878f4b0562 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:34.997307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddcd28-d16e-4684-9095-7b342ef1197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:34.998112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddcd28-d16e-4684-9095-7b342ef1197b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:34.998392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddcd28-d16e-4684-9095-7b342ef1197b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:34.998668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddcd28-d16e-4684-9095-7b342ef1197b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:35.002859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 371/752] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:34 2022] GET /identity/v3/role_assignments?user.id=a6e3bb6553264387936e9a534afc3ac0&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=433478cc318a467c931f13a757f161fa => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:35.003896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc027491-3769-4ff1-861b-22878f4b0562 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:35.022219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddcd28-d16e-4684-9095-7b342ef1197b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:35.023765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddcd28-d16e-4684-9095-7b342ef1197b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-b5ddcd28-d16e-4684-9095-7b342ef1197b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:35.057672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:35.058637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:35.060043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:35.060043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:35.060043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:35.060043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:35.060043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:35.060043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:35.060043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:35.060043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:35.060043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 382/753] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:35.064891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddcd28-d16e-4684-9095-7b342ef1197b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:35.069482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:35.171733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc027491-3769-4ff1-861b-22878f4b0562 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:35.243520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90a3a3-cc44-4719-8b97-c1afea1c1afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:35.244245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90a3a3-cc44-4719-8b97-c1afea1c1afa None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:35.244529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90a3a3-cc44-4719-8b97-c1afea1c1afa None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:35.244808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90a3a3-cc44-4719-8b97-c1afea1c1afa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:35.262441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90a3a3-cc44-4719-8b97-c1afea1c1afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-ad90a3a3-cc44-4719-8b97-c1afea1c1afa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:35.270296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:35.271285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:35.272133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:35.272775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 372/754] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:35.285757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90a3a3-cc44-4719-8b97-c1afea1c1afa None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:35.303972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddcd28-d16e-4684-9095-7b342ef1197b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:35.344719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:35.390649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b37f0b-57a6-4d07-9c77-938ec8cc96a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:35.391350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b37f0b-57a6-4d07-9c77-938ec8cc96a9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:35.391634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b37f0b-57a6-4d07-9c77-938ec8cc96a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:35.391909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b37f0b-57a6-4d07-9c77-938ec8cc96a9 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:35.392810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b37f0b-57a6-4d07-9c77-938ec8cc96a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:35.393097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b37f0b-57a6-4d07-9c77-938ec8cc96a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:35.394592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b37f0b-57a6-4d07-9c77-938ec8cc96a9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:35.428498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90a3a3-cc44-4719-8b97-c1afea1c1afa None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:35.429614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 383/755] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:35.447327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b37f0b-57a6-4d07-9c77-938ec8cc96a9 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:35.494457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751591c-031b-48e5-9aaa-ac62e07ee576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:35.496007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751591c-031b-48e5-9aaa-ac62e07ee576 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:35.497897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751591c-031b-48e5-9aaa-ac62e07ee576 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:35.498354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751591c-031b-48e5-9aaa-ac62e07ee576 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:35.499618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751591c-031b-48e5-9aaa-ac62e07ee576 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:35.500419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751591c-031b-48e5-9aaa-ac62e07ee576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:35.502926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751591c-031b-48e5-9aaa-ac62e07ee576 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:35.514010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 373/756] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:35.519058 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751591c-031b-48e5-9aaa-ac62e07ee576 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:35.602090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:35.684238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b37f0b-57a6-4d07-9c77-938ec8cc96a9 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:35.715076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:35.729372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751591c-031b-48e5-9aaa-ac62e07ee576 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:35.760040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb09223-ed62-4e69-bd4b-f94bb805a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:35.760627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09223-ed62-4e69-bd4b-f94bb805a610 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:35.760912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09223-ed62-4e69-bd4b-f94bb805a610 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:35.761183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09223-ed62-4e69-bd4b-f94bb805a610 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:35.762018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb09223-ed62-4e69-bd4b-f94bb805a610 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=2f73ecf1501a4ab181ad9a24ac54aec8) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:35.762328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb09223-ed62-4e69-bd4b-f94bb805a610 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:35.763801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb09223-ed62-4e69-bd4b-f94bb805a610 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:35.777028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 384/757] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:35 2022] GET /identity/v3/role_assignments?user.id=2f73ecf1501a4ab181ad9a24ac54aec8&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:35.799212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f486710-d3c5-4810-a2d0-edc8fb9978bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:35.799212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f486710-d3c5-4810-a2d0-edc8fb9978bc None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:35.799212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f486710-d3c5-4810-a2d0-edc8fb9978bc None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:35.799212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f486710-d3c5-4810-a2d0-edc8fb9978bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:35.954071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f486710-d3c5-4810-a2d0-edc8fb9978bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:35.955627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f486710-d3c5-4810-a2d0-edc8fb9978bc None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-9f486710-d3c5-4810-a2d0-edc8fb9978bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:35.973810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:35.975366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:35.976522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:35.976522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:35.976522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:35.976522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:35.976522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 374/758] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:35.987918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb09223-ed62-4e69-bd4b-f94bb805a610 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:36.130208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb09223-ed62-4e69-bd4b-f94bb805a610 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:36.170046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:36.180588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d79a33-5869-4054-9f73-70edf4c310d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:36.181281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d79a33-5869-4054-9f73-70edf4c310d6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:36.181566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d79a33-5869-4054-9f73-70edf4c310d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:36.181846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d79a33-5869-4054-9f73-70edf4c310d6 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:36.182590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d79a33-5869-4054-9f73-70edf4c310d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:36.182926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d79a33-5869-4054-9f73-70edf4c310d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:36.184363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d79a33-5869-4054-9f73-70edf4c310d6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:36.209615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 385/759] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:36.214795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f486710-d3c5-4810-a2d0-edc8fb9978bc None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:36.272395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781abec-96ca-4a8a-b14f-36d2efe8394e None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:36.273176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781abec-96ca-4a8a-b14f-36d2efe8394e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:36.273655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781abec-96ca-4a8a-b14f-36d2efe8394e None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:36.279146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 386/760] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:36.281023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f42ce-0f12-4640-b60a-4a22fa9f5d76 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:36.281506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f42ce-0f12-4640-b60a-4a22fa9f5d76 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:36.281987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f42ce-0f12-4640-b60a-4a22fa9f5d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:36.319906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5e7f42ce-0f12-4640-b60a-4a22fa9f5d76 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:36.321543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5e7f42ce-0f12-4640-b60a-4a22fa9f5d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:36.337558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f486710-d3c5-4810-a2d0-edc8fb9978bc None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:36.387887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5048a9-dfdc-4e8a-8e42-afb11d296031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:36.388611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5048a9-dfdc-4e8a-8e42-afb11d296031 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:36.388890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5048a9-dfdc-4e8a-8e42-afb11d296031 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:36.389165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5048a9-dfdc-4e8a-8e42-afb11d296031 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:36.407825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5048a9-dfdc-4e8a-8e42-afb11d296031 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:36.409280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5048a9-dfdc-4e8a-8e42-afb11d296031 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-ab5048a9-dfdc-4e8a-8e42-afb11d296031 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:36.437606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:36.438619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:36.439604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:36.442972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 375/761] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Jun 15 15:37:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:36.454219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5048a9-dfdc-4e8a-8e42-afb11d296031 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:36.458347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 387/762] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:36.471109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:36.488448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f42ce-0f12-4640-b60a-4a22fa9f5d76 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:36.656403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f42ce-0f12-4640-b60a-4a22fa9f5d76 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:36.680036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:36.691576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5048a9-dfdc-4e8a-8e42-afb11d296031 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:36.718349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816f7ea-0a4c-4494-85f5-fae03499b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:36.719188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816f7ea-0a4c-4494-85f5-fae03499b29e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:36.719469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816f7ea-0a4c-4494-85f5-fae03499b29e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:36.719745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816f7ea-0a4c-4494-85f5-fae03499b29e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:36.720404 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816f7ea-0a4c-4494-85f5-fae03499b29e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:36.721772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816f7ea-0a4c-4494-85f5-fae03499b29e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-3816f7ea-0a4c-4494-85f5-fae03499b29e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:36.750538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:36.751751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:36.752858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 388/763] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Jun 15 15:37:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:36.760373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3ba32-4893-4462-b49c-b8241d18db10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:36.761055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3ba32-4893-4462-b49c-b8241d18db10 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:36.761336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3ba32-4893-4462-b49c-b8241d18db10 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:36.761614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3ba32-4893-4462-b49c-b8241d18db10 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:36.762460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3ba32-4893-4462-b49c-b8241d18db10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:36.762789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3ba32-4893-4462-b49c-b8241d18db10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:36.764209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3ba32-4893-4462-b49c-b8241d18db10 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:36.766702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816f7ea-0a4c-4494-85f5-fae03499b29e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:36.813150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 376/764] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Jun 15 15:37:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:36.814425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3ba32-4893-4462-b49c-b8241d18db10 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:36.959677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:36.966035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816f7ea-0a4c-4494-85f5-fae03499b29e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.013065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbb74b-0cba-402e-97fe-87ef14dc106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.014518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbb74b-0cba-402e-97fe-87ef14dc106c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.014518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbb74b-0cba-402e-97fe-87ef14dc106c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.014518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbb74b-0cba-402e-97fe-87ef14dc106c None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.016647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3ba32-4893-4462-b49c-b8241d18db10 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.019527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbb74b-0cba-402e-97fe-87ef14dc106c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:37.020251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbb74b-0cba-402e-97fe-87ef14dc106c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.022259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbb74b-0cba-402e-97fe-87ef14dc106c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.037937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 389/765] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Jun 15 15:37:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:37.043905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbb74b-0cba-402e-97fe-87ef14dc106c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:37.063704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33601e49-f206-489d-8ccd-1c8e02dc1a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.064514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33601e49-f206-489d-8ccd-1c8e02dc1a54 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.064800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33601e49-f206-489d-8ccd-1c8e02dc1a54 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.065076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33601e49-f206-489d-8ccd-1c8e02dc1a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.079307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33601e49-f206-489d-8ccd-1c8e02dc1a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.081083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33601e49-f206-489d-8ccd-1c8e02dc1a54 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-33601e49-f206-489d-8ccd-1c8e02dc1a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:37.102790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:37.103725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:37.104612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:37.104612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:37.104612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:37.104612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:37.107211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 377/766] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:37.108955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26544b-bb6c-4c09-8c0f-0e2148a35825 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.109392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26544b-bb6c-4c09-8c0f-0e2148a35825 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.109809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26544b-bb6c-4c09-8c0f-0e2148a35825 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.117795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 378/767] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:37.117981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:37.118065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26544b-bb6c-4c09-8c0f-0e2148a35825 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:37.190279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbb74b-0cba-402e-97fe-87ef14dc106c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.241974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdcac2-696c-48c5-b670-6dbd276e2a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.242565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdcac2-696c-48c5-b670-6dbd276e2a48 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.242881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdcac2-696c-48c5-b670-6dbd276e2a48 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.243159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdcac2-696c-48c5-b670-6dbd276e2a48 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.243877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdcac2-696c-48c5-b670-6dbd276e2a48 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:37.244186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdcac2-696c-48c5-b670-6dbd276e2a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.245567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdcac2-696c-48c5-b670-6dbd276e2a48 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.255439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 390/768] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Jun 15 15:37:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:37.255919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cc004-0892-4e19-a196-7de7427f547d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.259812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cc004-0892-4e19-a196-7de7427f547d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.259812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cc004-0892-4e19-a196-7de7427f547d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.266163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26544b-bb6c-4c09-8c0f-0e2148a35825 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.280689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-7a0cc004-0892-4e19-a196-7de7427f547d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:37.282276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7a0cc004-0892-4e19-a196-7de7427f547d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:37.282792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:37.306463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e15a6-7073-44c4-8cb8-8f34b095cc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.307531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e15a6-7073-44c4-8cb8-8f34b095cc08 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.307824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e15a6-7073-44c4-8cb8-8f34b095cc08 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.308295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e15a6-7073-44c4-8cb8-8f34b095cc08 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.309438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e15a6-7073-44c4-8cb8-8f34b095cc08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:37.309969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e15a6-7073-44c4-8cb8-8f34b095cc08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.312179 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e15a6-7073-44c4-8cb8-8f34b095cc08 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.324964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 379/769] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:37.325819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e15a6-7073-44c4-8cb8-8f34b095cc08 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:37.373478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 391/770] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:37.374886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cc004-0892-4e19-a196-7de7427f547d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:37.440309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:37.442570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e15a6-7073-44c4-8cb8-8f34b095cc08 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.481281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512c9c1-ad34-4ad5-8a7e-112e7bed8c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.481945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512c9c1-ad34-4ad5-8a7e-112e7bed8c7c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.482228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512c9c1-ad34-4ad5-8a7e-112e7bed8c7c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.482507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512c9c1-ad34-4ad5-8a7e-112e7bed8c7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.483351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512c9c1-ad34-4ad5-8a7e-112e7bed8c7c None admin] RBAC: Adding query filter params (interface=public, service_id=6e9db5520cea4e0f851842e0199c98be, region_id=RegionOne) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:37.483663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512c9c1-ad34-4ad5-8a7e-112e7bed8c7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.485598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512c9c1-ad34-4ad5-8a7e-112e7bed8c7c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.496850 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 380/771] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Jun 15 15:37:37 2022] GET /identity/v3/endpoints?service_id=6e9db5520cea4e0f851842e0199c98be&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:37.496850 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512c9c1-ad34-4ad5-8a7e-112e7bed8c7c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:37.522938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cc004-0892-4e19-a196-7de7427f547d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.535586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:37.573320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f876a4-e093-4a19-ab5a-20242d674252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.574842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f876a4-e093-4a19-ab5a-20242d674252 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.575359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f876a4-e093-4a19-ab5a-20242d674252 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.575850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f876a4-e093-4a19-ab5a-20242d674252 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.586929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f876a4-e093-4a19-ab5a-20242d674252 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.589526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f876a4-e093-4a19-ab5a-20242d674252 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-57f876a4-e093-4a19-ab5a-20242d674252 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:37.611158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:37.612130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:37.613044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:37.613044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:37.613044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:37.613044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:37.613044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:37.613044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:37.613044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:37.613044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:37.616453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 392/772] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:37.617596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f876a4-e093-4a19-ab5a-20242d674252 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:37.623594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512c9c1-ad34-4ad5-8a7e-112e7bed8c7c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.661220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017264ca-6929-4608-af84-fb7333164fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.662609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017264ca-6929-4608-af84-fb7333164fe6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.662922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017264ca-6929-4608-af84-fb7333164fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.663198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017264ca-6929-4608-af84-fb7333164fe6 None admin] PATH_INFO: `/v3/services/network` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.663862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017264ca-6929-4608-af84-fb7333164fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.665641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017264ca-6929-4608-af84-fb7333164fe6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-017264ca-6929-4608-af84-fb7333164fe6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:37.677289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:37.678886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:37.684822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 381/773] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:37 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:37.685826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017264ca-6929-4608-af84-fb7333164fe6 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:37.735854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:37.739824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f876a4-e093-4a19-ab5a-20242d674252 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.780762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29149ae7-fec4-4f6a-98d8-4cae63e8e51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.781765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149ae7-fec4-4f6a-98d8-4cae63e8e51d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.782192 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149ae7-fec4-4f6a-98d8-4cae63e8e51d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.782589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149ae7-fec4-4f6a-98d8-4cae63e8e51d None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.783554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29149ae7-fec4-4f6a-98d8-4cae63e8e51d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.785502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29149ae7-fec4-4f6a-98d8-4cae63e8e51d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.788821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:37.798342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 393/774] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Jun 15 15:37:37 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:37.798539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017264ca-6929-4608-af84-fb7333164fe6 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.799012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a576094-cb58-4813-a469-5c8c8a89d916 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.799012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a576094-cb58-4813-a469-5c8c8a89d916 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.799012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a576094-cb58-4813-a469-5c8c8a89d916 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.799012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 394/775] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:37.799012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a576094-cb58-4813-a469-5c8c8a89d916 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:37.850199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa4c35-4b0c-4795-b729-d197291ad4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.851346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa4c35-4b0c-4795-b729-d197291ad4f5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.851637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa4c35-4b0c-4795-b729-d197291ad4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.851915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa4c35-4b0c-4795-b729-d197291ad4f5 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.853035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa4c35-4b0c-4795-b729-d197291ad4f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:37.853349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa4c35-4b0c-4795-b729-d197291ad4f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.855158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa4c35-4b0c-4795-b729-d197291ad4f5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.874919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 382/776] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:37.876487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d64ee-457c-4219-87fc-d7eb3be53836 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.876781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d64ee-457c-4219-87fc-d7eb3be53836 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.877072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d64ee-457c-4219-87fc-d7eb3be53836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.899895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b74d64ee-457c-4219-87fc-d7eb3be53836 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:37.901215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b74d64ee-457c-4219-87fc-d7eb3be53836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:37.926676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a576094-cb58-4813-a469-5c8c8a89d916 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:37.966683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f595a-4897-4639-b698-4aaa33e540b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:37.967916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f595a-4897-4639-b698-4aaa33e540b6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:37.968388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f595a-4897-4639-b698-4aaa33e540b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:37.968993 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f595a-4897-4639-b698-4aaa33e540b6 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:37.969896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f595a-4897-4639-b698-4aaa33e540b6 None admin] RBAC: Adding query filter params (name=network) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:37.970456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f595a-4897-4639-b698-4aaa33e540b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:37.972351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f595a-4897-4639-b698-4aaa33e540b6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:37.979975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 395/778] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:37:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:37.980188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 383/778] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:37.980916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d64ee-457c-4219-87fc-d7eb3be53836 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:37.984587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f595a-4897-4639-b698-4aaa33e540b6 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:38.023191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:38.077664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:38.089251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d64ee-457c-4219-87fc-d7eb3be53836 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:38.111060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f595a-4897-4639-b698-4aaa33e540b6 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:38.132956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124c104-58c7-41ae-90cb-e14767ace7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:38.133852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124c104-58c7-41ae-90cb-e14767ace7ea None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.134137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124c104-58c7-41ae-90cb-e14767ace7ea None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.134417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124c104-58c7-41ae-90cb-e14767ace7ea None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/4327e4eed5a1436ea1a00c2209bc5750/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.136529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124c104-58c7-41ae-90cb-e14767ace7ea None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=4327e4eed5a1436ea1a00c2209bc5750, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:38.137961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124c104-58c7-41ae-90cb-e14767ace7ea None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:38.150102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963f4ae-6472-4917-a425-18d863c2625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:38.150909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963f4ae-6472-4917-a425-18d863c2625c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.151271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963f4ae-6472-4917-a425-18d863c2625c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.151574 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963f4ae-6472-4917-a425-18d863c2625c None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.152911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963f4ae-6472-4917-a425-18d863c2625c None admin] RBAC: Adding query filter params (type=network) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:38.152911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963f4ae-6472-4917-a425-18d863c2625c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:38.153946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963f4ae-6472-4917-a425-18d863c2625c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:38.155202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 384/779] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:37:37 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/4327e4eed5a1436ea1a00c2209bc5750/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:37:38.156101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124c104-58c7-41ae-90cb-e14767ace7ea None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:38.168894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 396/780] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:37:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:38.175623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963f4ae-6472-4917-a425-18d863c2625c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:38.324032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124c104-58c7-41ae-90cb-e14767ace7ea None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:38.330065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:38.341284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963f4ae-6472-4917-a425-18d863c2625c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:38.362804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952875d0-efe8-401a-bb7e-22412f452b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:38.364224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952875d0-efe8-401a-bb7e-22412f452b77 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.364671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952875d0-efe8-401a-bb7e-22412f452b77 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.365137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952875d0-efe8-401a-bb7e-22412f452b77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.376749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952875d0-efe8-401a-bb7e-22412f452b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:38.380147 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee293076-4118-4e03-9868-3217e2820063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:38.381575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee293076-4118-4e03-9868-3217e2820063 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.381858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee293076-4118-4e03-9868-3217e2820063 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.382136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee293076-4118-4e03-9868-3217e2820063 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.382956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee293076-4118-4e03-9868-3217e2820063 None admin] RBAC: Adding query filter params (interface=public, service_id=48d4f9a92e5a42aea95267199c75f457, region_id=RegionOne) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:38.383274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee293076-4118-4e03-9868-3217e2820063 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:38.384634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee293076-4118-4e03-9868-3217e2820063 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-952875d0-efe8-401a-bb7e-22412f452b77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:38.385034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:38.385907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:38.386806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 385/781] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:38.393811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7077-f376-4f46-a775-d18fff8750b2 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.393811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7077-f376-4f46-a775-d18fff8750b2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.393811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7077-f376-4f46-a775-d18fff8750b2 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.394151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 397/782] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Jun 15 15:37:38 2022] GET /identity/v3/endpoints?service_id=48d4f9a92e5a42aea95267199c75f457&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:38.395087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 386/783] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:38.395779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee293076-4118-4e03-9868-3217e2820063 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:38.399986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7077-f376-4f46-a775-d18fff8750b2 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:38.416419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:38.526166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7077-f376-4f46-a775-d18fff8750b2 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:38.526518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee293076-4118-4e03-9868-3217e2820063 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:38.565857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a294573-9e3a-4a18-b1d8-c135becc7498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:38.566560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a294573-9e3a-4a18-b1d8-c135becc7498 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.566880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a294573-9e3a-4a18-b1d8-c135becc7498 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.567164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a294573-9e3a-4a18-b1d8-c135becc7498 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.567506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933d4fc-65c3-489c-8e64-698610a5f71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:38.567876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a294573-9e3a-4a18-b1d8-c135becc7498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:38.568847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933d4fc-65c3-489c-8e64-698610a5f71a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.569208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933d4fc-65c3-489c-8e64-698610a5f71a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.569547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933d4fc-65c3-489c-8e64-698610a5f71a None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.570361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933d4fc-65c3-489c-8e64-698610a5f71a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:38.570850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933d4fc-65c3-489c-8e64-698610a5f71a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:38.571063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a294573-9e3a-4a18-b1d8-c135becc7498 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:38.572558 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933d4fc-65c3-489c-8e64-698610a5f71a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:38.584852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 398/785] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:38.586176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958decb-2545-44f7-a00e-69f9cb044ae8 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.586534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958decb-2545-44f7-a00e-69f9cb044ae8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.586909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958decb-2545-44f7-a00e-69f9cb044ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.589326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 387/785] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Jun 15 15:37:38 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:38.593605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a294573-9e3a-4a18-b1d8-c135becc7498 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:38.605419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e958decb-2545-44f7-a00e-69f9cb044ae8 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:38.606702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e958decb-2545-44f7-a00e-69f9cb044ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:38.618841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:38.682104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:38.690321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 399/786] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:38.713286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958decb-2545-44f7-a00e-69f9cb044ae8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:38.714814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a294573-9e3a-4a18-b1d8-c135becc7498 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:38.754171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f43039-b94f-467a-92f5-0ff6cfd80257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:38.755562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f43039-b94f-467a-92f5-0ff6cfd80257 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.756059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f43039-b94f-467a-92f5-0ff6cfd80257 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.756539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f43039-b94f-467a-92f5-0ff6cfd80257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.770898 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f43039-b94f-467a-92f5-0ff6cfd80257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:38.773413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f43039-b94f-467a-92f5-0ff6cfd80257 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-a7f43039-b94f-467a-92f5-0ff6cfd80257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:38.794001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:38.795031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:38.795948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:38.795948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:38.795948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:38.795948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:38.798456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 388/787] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:38.804752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f43039-b94f-467a-92f5-0ff6cfd80257 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:38.821998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958decb-2545-44f7-a00e-69f9cb044ae8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:38.859632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15516431-2d9f-456c-b13b-38f81f135ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:38.860205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15516431-2d9f-456c-b13b-38f81f135ee3 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.860486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15516431-2d9f-456c-b13b-38f81f135ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.860760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15516431-2d9f-456c-b13b-38f81f135ee3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:38.871877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15516431-2d9f-456c-b13b-38f81f135ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-15516431-2d9f-456c-b13b-38f81f135ee3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:38.879826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:38.880796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:38.881658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 400/788] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:38.887127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15516431-2d9f-456c-b13b-38f81f135ee3 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:38.898408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:38.945097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f43039-b94f-467a-92f5-0ff6cfd80257 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:38.978501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:38.996807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636d0c1-b2c7-46fe-97c5-1820a42ebcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:38.998113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636d0c1-b2c7-46fe-97c5-1820a42ebcb2 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:38.998923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636d0c1-b2c7-46fe-97c5-1820a42ebcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:38.999833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636d0c1-b2c7-46fe-97c5-1820a42ebcb2 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:39.001049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636d0c1-b2c7-46fe-97c5-1820a42ebcb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:39.001564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636d0c1-b2c7-46fe-97c5-1820a42ebcb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:39.002416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15516431-2d9f-456c-b13b-38f81f135ee3 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:39.003526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636d0c1-b2c7-46fe-97c5-1820a42ebcb2 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:39.021456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 389/789] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:39.027935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636d0c1-b2c7-46fe-97c5-1820a42ebcb2 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:39.095618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1eda6-cf54-47c8-bef8-1d1e667ff8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:39.096134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1eda6-cf54-47c8-bef8-1d1e667ff8b2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:39.096419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1eda6-cf54-47c8-bef8-1d1e667ff8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:39.096695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1eda6-cf54-47c8-bef8-1d1e667ff8b2 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:39.097452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1eda6-cf54-47c8-bef8-1d1e667ff8b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:39.097830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1eda6-cf54-47c8-bef8-1d1e667ff8b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:39.099356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1eda6-cf54-47c8-bef8-1d1e667ff8b2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:39.118389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 401/790] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:39.123458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1eda6-cf54-47c8-bef8-1d1e667ff8b2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:39.149026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636d0c1-b2c7-46fe-97c5-1820a42ebcb2 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:39.190277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45142997-f983-4cbc-a4ab-fe013400c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:39.192166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45142997-f983-4cbc-a4ab-fe013400c8eb None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:39.193374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45142997-f983-4cbc-a4ab-fe013400c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:39.193885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45142997-f983-4cbc-a4ab-fe013400c8eb None admin] PATH_INFO: `/v3/users/swift` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:39.203338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45142997-f983-4cbc-a4ab-fe013400c8eb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:39.205606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45142997-f983-4cbc-a4ab-fe013400c8eb None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-45142997-f983-4cbc-a4ab-fe013400c8eb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:39.224821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:39.225914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:39.226921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:39.230046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 390/791] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:39.236478 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45142997-f983-4cbc-a4ab-fe013400c8eb None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:39.297841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:39.304661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:39.365741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45142997-f983-4cbc-a4ab-fe013400c8eb None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:39.389597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1eda6-cf54-47c8-bef8-1d1e667ff8b2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:39.407104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464092a7-8866-4629-8f44-6b15a86c3364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:39.408144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464092a7-8866-4629-8f44-6b15a86c3364 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:39.408687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464092a7-8866-4629-8f44-6b15a86c3364 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:39.409172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464092a7-8866-4629-8f44-6b15a86c3364 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:39.410244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464092a7-8866-4629-8f44-6b15a86c3364 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:39.410786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464092a7-8866-4629-8f44-6b15a86c3364 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:39.412478 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464092a7-8866-4629-8f44-6b15a86c3364 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:39.433489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 391/792] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Jun 15 15:37:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:39.439538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464092a7-8866-4629-8f44-6b15a86c3364 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:39.527472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922902f1-d07a-4897-8f51-c066acb2c2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:39.528287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922902f1-d07a-4897-8f51-c066acb2c2d5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:39.528573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922902f1-d07a-4897-8f51-c066acb2c2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:39.528847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922902f1-d07a-4897-8f51-c066acb2c2d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:39.555580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922902f1-d07a-4897-8f51-c066acb2c2d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:39.557060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922902f1-d07a-4897-8f51-c066acb2c2d5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:39.559923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464092a7-8866-4629-8f44-6b15a86c3364 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:39.562597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-922902f1-d07a-4897-8f51-c066acb2c2d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:39.592101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:39.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:39.594000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:39.594000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:39.594000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:39.594000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:39.594000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 402/793] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:39.598753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922902f1-d07a-4897-8f51-c066acb2c2d5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:39.603854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc6427-d02d-4fea-ad37-bb507b6a3d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:39.605289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc6427-d02d-4fea-ad37-bb507b6a3d51 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:39.605858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc6427-d02d-4fea-ad37-bb507b6a3d51 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:39.607312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc6427-d02d-4fea-ad37-bb507b6a3d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:39.619790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc6427-d02d-4fea-ad37-bb507b6a3d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:39.622036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc6427-d02d-4fea-ad37-bb507b6a3d51 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-02dc6427-d02d-4fea-ad37-bb507b6a3d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:39.643698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:39.644882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:39.645785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:39.645785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:39.645785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:39.645785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:39.648986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 392/794] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:39.654808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc6427-d02d-4fea-ad37-bb507b6a3d51 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:39.773064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc6427-d02d-4fea-ad37-bb507b6a3d51 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:39.813253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d59bb0-f0ca-4177-9644-f8748963de45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:39.814293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d59bb0-f0ca-4177-9644-f8748963de45 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:39.814849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d59bb0-f0ca-4177-9644-f8748963de45 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:39.815347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d59bb0-f0ca-4177-9644-f8748963de45 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:39.816626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d59bb0-f0ca-4177-9644-f8748963de45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:39.817188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d59bb0-f0ca-4177-9644-f8748963de45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:39.819040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d59bb0-f0ca-4177-9644-f8748963de45 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:39.832116 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 393/795] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:39.838187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d59bb0-f0ca-4177-9644-f8748963de45 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:39.858640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:39.892674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922902f1-d07a-4897-8f51-c066acb2c2d5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:39.947209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:39.952330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d59bb0-f0ca-4177-9644-f8748963de45 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:39.979592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62388da-519d-41d1-9bab-8d82ac6f435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:39.980285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62388da-519d-41d1-9bab-8d82ac6f435c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:39.980642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62388da-519d-41d1-9bab-8d82ac6f435c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:39.980839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62388da-519d-41d1-9bab-8d82ac6f435c None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:39.981571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62388da-519d-41d1-9bab-8d82ac6f435c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:39.982141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62388da-519d-41d1-9bab-8d82ac6f435c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:39.983291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62388da-519d-41d1-9bab-8d82ac6f435c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:39.993197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f1643-a661-4043-8d90-8fc826eff99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:39.995096 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f1643-a661-4043-8d90-8fc826eff99b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:39.995590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f1643-a661-4043-8d90-8fc826eff99b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:39.996072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f1643-a661-4043-8d90-8fc826eff99b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.008733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 403/796] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:40.010399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f1643-a661-4043-8d90-8fc826eff99b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:40.012101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f1643-a661-4043-8d90-8fc826eff99b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:40.014041 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62388da-519d-41d1-9bab-8d82ac6f435c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-c47f1643-a661-4043-8d90-8fc826eff99b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:40.044424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:40.045406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:40.046302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:40.046302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:40.046302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:40.046302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:40.046302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:40.046302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:40.046302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:40.046302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:40.059359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 394/797] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:40.062252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f1643-a661-4043-8d90-8fc826eff99b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:40.209166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62388da-519d-41d1-9bab-8d82ac6f435c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:40.218874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:40.261684 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f1643-a661-4043-8d90-8fc826eff99b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:40.284141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33a833-f590-4962-891d-0dc25606e4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:40.285402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33a833-f590-4962-891d-0dc25606e4d1 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.285891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33a833-f590-4962-891d-0dc25606e4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.286402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33a833-f590-4962-891d-0dc25606e4d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.303800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33a833-f590-4962-891d-0dc25606e4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:40.305288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33a833-f590-4962-891d-0dc25606e4d1 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:40.330034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4917e-7acb-431c-8d92-a5d8e3efdcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:40.331090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4917e-7acb-431c-8d92-a5d8e3efdcb6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.331609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4917e-7acb-431c-8d92-a5d8e3efdcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.332092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4917e-7acb-431c-8d92-a5d8e3efdcb6 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.333147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4917e-7acb-431c-8d92-a5d8e3efdcb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:40.334202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4917e-7acb-431c-8d92-a5d8e3efdcb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:40.335879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4917e-7acb-431c-8d92-a5d8e3efdcb6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-0e33a833-f590-4962-891d-0dc25606e4d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:40.340663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:40.341696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:40.342650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:40.347501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 404/798] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:37:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:40.349629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2f267-dbaa-4654-b4db-ff379fe0a488 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.350193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2f267-dbaa-4654-b4db-ff379fe0a488 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.350695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2f267-dbaa-4654-b4db-ff379fe0a488 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.353011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 405/799] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:40.353911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2f267-dbaa-4654-b4db-ff379fe0a488 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:40.360990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 395/800] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:40.362499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacf7f1-ce73-43b1-ad2f-16791f6a2027 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.367653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacf7f1-ce73-43b1-ad2f-16791f6a2027 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.368759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacf7f1-ce73-43b1-ad2f-16791f6a2027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.386412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-cfacf7f1-ce73-43b1-ad2f-16791f6a2027 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:40.388182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cfacf7f1-ce73-43b1-ad2f-16791f6a2027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:40.401241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:40.465365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2f267-dbaa-4654-b4db-ff379fe0a488 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:40.468949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 396/801] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:40.469972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacf7f1-ce73-43b1-ad2f-16791f6a2027 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:40.517689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fef26c-237c-409e-844a-8ee40f3b3fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:40.518371 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef26c-237c-409e-844a-8ee40f3b3fd3 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.518655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef26c-237c-409e-844a-8ee40f3b3fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.518980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef26c-237c-409e-844a-8ee40f3b3fd3 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.520076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef26c-237c-409e-844a-8ee40f3b3fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:40.520385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef26c-237c-409e-844a-8ee40f3b3fd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:40.522368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef26c-237c-409e-844a-8ee40f3b3fd3 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:40.540660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 406/802] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Jun 15 15:37:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:40.541503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fef26c-237c-409e-844a-8ee40f3b3fd3 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:40.543131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:40.585807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacf7f1-ce73-43b1-ad2f-16791f6a2027 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:40.632120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5a63f-d4aa-4f9a-b6ac-40eb9517c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:40.633012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5a63f-d4aa-4f9a-b6ac-40eb9517c590 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.633295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5a63f-d4aa-4f9a-b6ac-40eb9517c590 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.633601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5a63f-d4aa-4f9a-b6ac-40eb9517c590 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/a6e3bb6553264387936e9a534afc3ac0/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.635594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5a63f-d4aa-4f9a-b6ac-40eb9517c590 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=a6e3bb6553264387936e9a534afc3ac0, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:40.637134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5a63f-d4aa-4f9a-b6ac-40eb9517c590 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:40.655001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 397/803] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:37:40 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/a6e3bb6553264387936e9a534afc3ac0/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:37:40.655001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5a63f-d4aa-4f9a-b6ac-40eb9517c590 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:40.661403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:40.667162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fef26c-237c-409e-844a-8ee40f3b3fd3 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:40.718062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f27f5-133d-4d78-a264-15431bf64cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:40.719899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f27f5-133d-4d78-a264-15431bf64cd9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.720380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f27f5-133d-4d78-a264-15431bf64cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.720800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f27f5-133d-4d78-a264-15431bf64cd9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.721825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f27f5-133d-4d78-a264-15431bf64cd9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:40.724499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f27f5-133d-4d78-a264-15431bf64cd9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-325f27f5-133d-4d78-a264-15431bf64cd9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:40.742895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:40.744097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 15:37:40.745143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:40.748767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 407/804] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Jun 15 15:37:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:40.750926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358bcd0-ed05-4503-85f7-b5d25728f6ff None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.751366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358bcd0-ed05-4503-85f7-b5d25728f6ff None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.752608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358bcd0-ed05-4503-85f7-b5d25728f6ff None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.754513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 408/805] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:40.756039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358bcd0-ed05-4503-85f7-b5d25728f6ff None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:40.794370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5a63f-d4aa-4f9a-b6ac-40eb9517c590 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:40.830410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:40.840722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022234a-9d38-426c-8c9d-982bb7496ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:40.842255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022234a-9d38-426c-8c9d-982bb7496ee8 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.842810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022234a-9d38-426c-8c9d-982bb7496ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.843405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022234a-9d38-426c-8c9d-982bb7496ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.856106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022234a-9d38-426c-8c9d-982bb7496ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:40.857553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022234a-9d38-426c-8c9d-982bb7496ee8 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-0022234a-9d38-426c-8c9d-982bb7496ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:40.877800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:40.878840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:40.879983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:40.879983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:40.879983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:40.879983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:40.879983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 398/806] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:40.880599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d0dc8-c0c7-4584-ab79-96dfc0511df5 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.880882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d0dc8-c0c7-4584-ab79-96dfc0511df5 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.881244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d0dc8-c0c7-4584-ab79-96dfc0511df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.890849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358bcd0-ed05-4503-85f7-b5d25728f6ff None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:40.903144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ce1d0dc8-c0c7-4584-ab79-96dfc0511df5 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:40.904331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ce1d0dc8-c0c7-4584-ab79-96dfc0511df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:40.934588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb19ed-136f-46ad-ac92-d4588a88a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:40.935446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb19ed-136f-46ad-ac92-d4588a88a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:40.936025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb19ed-136f-46ad-ac92-d4588a88a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:40.936305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb19ed-136f-46ad-ac92-d4588a88a4d4 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:40.937767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb19ed-136f-46ad-ac92-d4588a88a4d4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:40.938083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb19ed-136f-46ad-ac92-d4588a88a4d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:40.939905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb19ed-136f-46ad-ac92-d4588a88a4d4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:40.946254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 409/807] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Jun 15 15:37:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:40.947143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb19ed-136f-46ad-ac92-d4588a88a4d4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:40.986833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 399/808] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:40.986833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d0dc8-c0c7-4584-ab79-96dfc0511df5 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:40.991243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:41.086095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d0dc8-c0c7-4584-ab79-96dfc0511df5 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.092895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb19ed-136f-46ad-ac92-d4588a88a4d4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.097375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:41.126179 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796d0fc-775c-46f3-bdff-43c2ca57aae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.127097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796d0fc-775c-46f3-bdff-43c2ca57aae7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.127455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796d0fc-775c-46f3-bdff-43c2ca57aae7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.127809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796d0fc-775c-46f3-bdff-43c2ca57aae7 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.128705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796d0fc-775c-46f3-bdff-43c2ca57aae7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:41.129086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796d0fc-775c-46f3-bdff-43c2ca57aae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:41.130573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796d0fc-775c-46f3-bdff-43c2ca57aae7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:41.140830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 400/809] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Jun 15 15:37:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:41.140830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796d0fc-775c-46f3-bdff-43c2ca57aae7 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:41.142370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03572b96-ffe4-41b1-8235-fd6bb893fd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.143381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03572b96-ffe4-41b1-8235-fd6bb893fd6f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.143849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03572b96-ffe4-41b1-8235-fd6bb893fd6f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.144308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03572b96-ffe4-41b1-8235-fd6bb893fd6f None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.145475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03572b96-ffe4-41b1-8235-fd6bb893fd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:41.146553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03572b96-ffe4-41b1-8235-fd6bb893fd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:41.148282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03572b96-ffe4-41b1-8235-fd6bb893fd6f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:41.162669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 410/810] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:41.163977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03572b96-ffe4-41b1-8235-fd6bb893fd6f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:41.248093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796d0fc-775c-46f3-bdff-43c2ca57aae7 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.248301 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:41.264589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03572b96-ffe4-41b1-8235-fd6bb893fd6f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.290191 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b7565-8f26-4776-aa45-d9ed3d81af8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.291020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7565-8f26-4776-aa45-d9ed3d81af8d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.291306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7565-8f26-4776-aa45-d9ed3d81af8d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.291578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7565-8f26-4776-aa45-d9ed3d81af8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.299553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1fe8b-e41a-4b97-b47c-bbd45dbc657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.300239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1fe8b-e41a-4b97-b47c-bbd45dbc657c None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.300523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1fe8b-e41a-4b97-b47c-bbd45dbc657c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.300802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1fe8b-e41a-4b97-b47c-bbd45dbc657c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.301536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1fe8b-e41a-4b97-b47c-bbd45dbc657c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:41.303479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1fe8b-e41a-4b97-b47c-bbd45dbc657c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:41.303598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b7565-8f26-4776-aa45-d9ed3d81af8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-841b7565-8f26-4776-aa45-d9ed3d81af8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:41.311430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:41.312522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:41.313572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:41.315646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 401/811] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:41.316874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216d90-ad56-46ff-92de-90e634af34dd None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.317164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216d90-ad56-46ff-92de-90e634af34dd None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.317447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216d90-ad56-46ff-92de-90e634af34dd None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.318970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 402/812] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:41.319453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5216d90-ad56-46ff-92de-90e634af34dd None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:41.335395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 411/813] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Jun 15 15:37:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:41.337081 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469f834-674c-4298-b746-1d2e518f80aa None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.337445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469f834-674c-4298-b746-1d2e518f80aa None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.337775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469f834-674c-4298-b746-1d2e518f80aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.356323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2469f834-674c-4298-b746-1d2e518f80aa None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:41.357543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2469f834-674c-4298-b746-1d2e518f80aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:41.394811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:41.469437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 412/814] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:41.470825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469f834-674c-4298-b746-1d2e518f80aa None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:41.503083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5216d90-ad56-46ff-92de-90e634af34dd None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.543024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a354d1-83b9-45d9-8ce4-3213f9f55390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.543817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a354d1-83b9-45d9-8ce4-3213f9f55390 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.544101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a354d1-83b9-45d9-8ce4-3213f9f55390 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.544375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a354d1-83b9-45d9-8ce4-3213f9f55390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.552719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a354d1-83b9-45d9-8ce4-3213f9f55390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:41.554153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a354d1-83b9-45d9-8ce4-3213f9f55390 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:41.557275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-87a354d1-83b9-45d9-8ce4-3213f9f55390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:41.570120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:41.571356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:41.572460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:41.572460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:41.572460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:41.572460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:41.572460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:41.572460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:41.572460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:41.572460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:41.577733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 403/815] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:41.577733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a354d1-83b9-45d9-8ce4-3213f9f55390 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:41.598183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469f834-674c-4298-b746-1d2e518f80aa None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.641949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae51f2-0aae-4825-921b-a1d417096add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.642708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae51f2-0aae-4825-921b-a1d417096add None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.643040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae51f2-0aae-4825-921b-a1d417096add None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.643323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae51f2-0aae-4825-921b-a1d417096add None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.644232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae51f2-0aae-4825-921b-a1d417096add None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:41.645183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae51f2-0aae-4825-921b-a1d417096add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:41.647419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae51f2-0aae-4825-921b-a1d417096add None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:41.658744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 413/816] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:41.659838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae51f2-0aae-4825-921b-a1d417096add None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:41.693566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:41.703874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a354d1-83b9-45d9-8ce4-3213f9f55390 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.742497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc0091-0434-4289-9f01-cd1af2e8d20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.743611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc0091-0434-4289-9f01-cd1af2e8d20e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.743995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc0091-0434-4289-9f01-cd1af2e8d20e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.744391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc0091-0434-4289-9f01-cd1af2e8d20e None admin] PATH_INFO: `/v3/services/network` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.745151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc0091-0434-4289-9f01-cd1af2e8d20e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:41.746709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc0091-0434-4289-9f01-cd1af2e8d20e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-46bc0091-0434-4289-9f01-cd1af2e8d20e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:41.759843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:41.761045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 15:37:41.762054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:41.763732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 404/817] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:41 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:41.764636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc0091-0434-4289-9f01-cd1af2e8d20e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:41.773047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae51f2-0aae-4825-921b-a1d417096add None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.802702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:41.808970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260b994-3dba-4f45-9f06-8d333e0819a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.809655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260b994-3dba-4f45-9f06-8d333e0819a1 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.809944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260b994-3dba-4f45-9f06-8d333e0819a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.810226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260b994-3dba-4f45-9f06-8d333e0819a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.811092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260b994-3dba-4f45-9f06-8d333e0819a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:41.811409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260b994-3dba-4f45-9f06-8d333e0819a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:41.813032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260b994-3dba-4f45-9f06-8d333e0819a1 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:41.827652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 414/818] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:41.828542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260b994-3dba-4f45-9f06-8d333e0819a1 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:41.880026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc0091-0434-4289-9f01-cd1af2e8d20e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.918349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b530-9a64-49d7-b8e3-3a4b8f4cd8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.919953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b530-9a64-49d7-b8e3-3a4b8f4cd8c1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.920390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b530-9a64-49d7-b8e3-3a4b8f4cd8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.921075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b530-9a64-49d7-b8e3-3a4b8f4cd8c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.932036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260b994-3dba-4f45-9f06-8d333e0819a1 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:41.936570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752b530-9a64-49d7-b8e3-3a4b8f4cd8c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:41.938186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752b530-9a64-49d7-b8e3-3a4b8f4cd8c1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-4752b530-9a64-49d7-b8e3-3a4b8f4cd8c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:41.962232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:41.963386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:41.964483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:41.964483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:41.964483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:41.964483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:41.966871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 405/819] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:41.968456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b530-9a64-49d7-b8e3-3a4b8f4cd8c1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:41.972342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2122fd-1516-45e4-8c2f-85794bfb3090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:41.973122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2122fd-1516-45e4-8c2f-85794bfb3090 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:41.973527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2122fd-1516-45e4-8c2f-85794bfb3090 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:41.973942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2122fd-1516-45e4-8c2f-85794bfb3090 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:41.974784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2122fd-1516-45e4-8c2f-85794bfb3090 None admin] RBAC: Adding query filter params (name=network) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:41.975208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2122fd-1516-45e4-8c2f-85794bfb3090 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:41.976695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2122fd-1516-45e4-8c2f-85794bfb3090 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:41.986207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 415/820] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:37:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:41.986398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:41.986501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2122fd-1516-45e4-8c2f-85794bfb3090 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:42.046369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:42.093496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2122fd-1516-45e4-8c2f-85794bfb3090 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:42.112839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b530-9a64-49d7-b8e3-3a4b8f4cd8c1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:42.129308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3d6f-0e08-43b0-a252-ee7b48e37999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:42.130751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3d6f-0e08-43b0-a252-ee7b48e37999 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:42.131178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3d6f-0e08-43b0-a252-ee7b48e37999 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:42.131556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3d6f-0e08-43b0-a252-ee7b48e37999 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:42.132487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3d6f-0e08-43b0-a252-ee7b48e37999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:42.132898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3d6f-0e08-43b0-a252-ee7b48e37999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:42.134375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3d6f-0e08-43b0-a252-ee7b48e37999 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:42.148394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 416/821] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:42.149439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3d6f-0e08-43b0-a252-ee7b48e37999 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:42.157363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb9667-e87f-4bce-8fec-bf8112b19868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:42.158522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9667-e87f-4bce-8fec-bf8112b19868 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:42.158988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9667-e87f-4bce-8fec-bf8112b19868 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:42.159953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9667-e87f-4bce-8fec-bf8112b19868 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/2f73ecf1501a4ab181ad9a24ac54aec8/roles/01737d3768e2407780ff51b345e1d020` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:42.179811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb9667-e87f-4bce-8fec-bf8112b19868 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=2f73ecf1501a4ab181ad9a24ac54aec8, role_id=01737d3768e2407780ff51b345e1d020)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:42.182194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb9667-e87f-4bce-8fec-bf8112b19868 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:42.201967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 406/822] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:37:41 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/2f73ecf1501a4ab181ad9a24ac54aec8/roles/01737d3768e2407780ff51b345e1d020 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:37:42.203362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb9667-e87f-4bce-8fec-bf8112b19868 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:42.324825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3d6f-0e08-43b0-a252-ee7b48e37999 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:42.340925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:42.346835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:42.376827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130a8a7-918c-4e1f-b8cb-b48d5ed85e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:42.377773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130a8a7-918c-4e1f-b8cb-b48d5ed85e68 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:42.378160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130a8a7-918c-4e1f-b8cb-b48d5ed85e68 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:42.378543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130a8a7-918c-4e1f-b8cb-b48d5ed85e68 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:42.379509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130a8a7-918c-4e1f-b8cb-b48d5ed85e68 None admin] RBAC: Adding query filter params (type=network) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:42.379942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130a8a7-918c-4e1f-b8cb-b48d5ed85e68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:42.381999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130a8a7-918c-4e1f-b8cb-b48d5ed85e68 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:42.394303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 417/823] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:37:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:42.401925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130a8a7-918c-4e1f-b8cb-b48d5ed85e68 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:42.403962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb9667-e87f-4bce-8fec-bf8112b19868 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:42.443444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9876a-09ef-4b00-8492-847f0756c72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:42.444649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9876a-09ef-4b00-8492-847f0756c72a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:42.445072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9876a-09ef-4b00-8492-847f0756c72a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:42.445484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9876a-09ef-4b00-8492-847f0756c72a None admin] PATH_INFO: `/v3/users/nova` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:42.454441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9876a-09ef-4b00-8492-847f0756c72a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:42.456095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9876a-09ef-4b00-8492-847f0756c72a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-45a9876a-09ef-4b00-8492-847f0756c72a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:42.473922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:42.475637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:37:42.476781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:42.479280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 407/824] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Jun 15 15:37:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:42.484731 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9876a-09ef-4b00-8492-847f0756c72a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:42.539295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130a8a7-918c-4e1f-b8cb-b48d5ed85e68 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:42.575835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077249d2-7b04-48f7-9af8-0737334df738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:42.577133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077249d2-7b04-48f7-9af8-0737334df738 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:42.577557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077249d2-7b04-48f7-9af8-0737334df738 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:42.577971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077249d2-7b04-48f7-9af8-0737334df738 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:42.579020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077249d2-7b04-48f7-9af8-0737334df738 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:42.580586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077249d2-7b04-48f7-9af8-0737334df738 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:42.621018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 418/825] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Jun 15 15:37:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:37:42.625478 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9876a-09ef-4b00-8492-847f0756c72a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:42.653189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:42.680821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa86c2-46ef-4fe8-81d0-601247931882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:42.681505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa86c2-46ef-4fe8-81d0-601247931882 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:42.681789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa86c2-46ef-4fe8-81d0-601247931882 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:42.682066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa86c2-46ef-4fe8-81d0-601247931882 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:42.682941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa86c2-46ef-4fe8-81d0-601247931882 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:42.683252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa86c2-46ef-4fe8-81d0-601247931882 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:42.684760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa86c2-46ef-4fe8-81d0-601247931882 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:42.705508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 408/826] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Jun 15 15:37:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:42.711306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077249d2-7b04-48f7-9af8-0737334df738 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:42.765784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:42.852938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077249d2-7b04-48f7-9af8-0737334df738 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:42.895976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28606dd1-d514-4680-a37b-f83c038964bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:42.896655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606dd1-d514-4680-a37b-f83c038964bd None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:42.896934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606dd1-d514-4680-a37b-f83c038964bd None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:42.897208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606dd1-d514-4680-a37b-f83c038964bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:42.945226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3dc42-b336-4e80-888a-ff73326ce48e None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:42.945579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3dc42-b336-4e80-888a-ff73326ce48e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:42.945771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3dc42-b336-4e80-888a-ff73326ce48e None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:42.947490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 409/827] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:42.952985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b357e1-706d-4701-aacc-6fa679543c88 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:42.953274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b357e1-706d-4701-aacc-6fa679543c88 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:42.953541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b357e1-706d-4701-aacc-6fa679543c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:42.980784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-32b357e1-706d-4701-aacc-6fa679543c88 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:42.982197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-32b357e1-706d-4701-aacc-6fa679543c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:43.011551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28606dd1-d514-4680-a37b-f83c038964bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:43.013624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28606dd1-d514-4680-a37b-f83c038964bd None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:43.055740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:43.065896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 410/828] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:43.088746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b357e1-706d-4701-aacc-6fa679543c88 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:43.146048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 419/829] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:37:43.151679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28606dd1-d514-4680-a37b-f83c038964bd None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:43.216972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b357e1-706d-4701-aacc-6fa679543c88 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:43.218520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:43.263661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a2343-c9a9-4725-ac77-1dd8606d1b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:43.264759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a2343-c9a9-4725-ac77-1dd8606d1b28 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.265047 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a2343-c9a9-4725-ac77-1dd8606d1b28 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.265325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a2343-c9a9-4725-ac77-1dd8606d1b28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:43.279178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a2343-c9a9-4725-ac77-1dd8606d1b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:43.281567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28606dd1-d514-4680-a37b-f83c038964bd None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-f64a2343-c9a9-4725-ac77-1dd8606d1b28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:43.288990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:43.289999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:43.291990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 411/830] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:43.296808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a2343-c9a9-4725-ac77-1dd8606d1b28 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:43.327645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c448256-b7c0-4252-97ed-3505e2d2cfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:43.328871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c448256-b7c0-4252-97ed-3505e2d2cfc0 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.329369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c448256-b7c0-4252-97ed-3505e2d2cfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.329873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c448256-b7c0-4252-97ed-3505e2d2cfc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:43.332202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:43.342965 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c448256-b7c0-4252-97ed-3505e2d2cfc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:43.344733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c448256-b7c0-4252-97ed-3505e2d2cfc0 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-8c448256-b7c0-4252-97ed-3505e2d2cfc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:43.363267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:43.364201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:43.365722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:43.365722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:43.365722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:43.365722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:43.368110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 420/831] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:43.374439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c448256-b7c0-4252-97ed-3505e2d2cfc0 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:43.424595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a2343-c9a9-4725-ac77-1dd8606d1b28 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:43.467258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad115a-cfd5-4428-8291-33eac7daabe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:43.468286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad115a-cfd5-4428-8291-33eac7daabe2 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.468833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad115a-cfd5-4428-8291-33eac7daabe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.469318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad115a-cfd5-4428-8291-33eac7daabe2 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:43.470272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad115a-cfd5-4428-8291-33eac7daabe2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:43.470814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad115a-cfd5-4428-8291-33eac7daabe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:43.472413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad115a-cfd5-4428-8291-33eac7daabe2 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:43.482219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 412/832] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:43.487336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad115a-cfd5-4428-8291-33eac7daabe2 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:43.512855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c448256-b7c0-4252-97ed-3505e2d2cfc0 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:43.514102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:43.552830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f54208-d908-462f-836d-a7ea39de83ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:43.553496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f54208-d908-462f-836d-a7ea39de83ad None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.553772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f54208-d908-462f-836d-a7ea39de83ad None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.554044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f54208-d908-462f-836d-a7ea39de83ad None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:43.554774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f54208-d908-462f-836d-a7ea39de83ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:43.555082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f54208-d908-462f-836d-a7ea39de83ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:43.556576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f54208-d908-462f-836d-a7ea39de83ad None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:43.573468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 421/833] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:43.574860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f54208-d908-462f-836d-a7ea39de83ad None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:43.614305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad115a-cfd5-4428-8291-33eac7daabe2 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:43.629444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:43.660478 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924cd059-f6c4-4599-a45a-e222e78dc0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:43.661784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cd059-f6c4-4599-a45a-e222e78dc0ed None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.662069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cd059-f6c4-4599-a45a-e222e78dc0ed None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.663119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cd059-f6c4-4599-a45a-e222e78dc0ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:43.673702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924cd059-f6c4-4599-a45a-e222e78dc0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:43.675291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924cd059-f6c4-4599-a45a-e222e78dc0ed None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-924cd059-f6c4-4599-a45a-e222e78dc0ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:43.693730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:43.694677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:43.695640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:43.695640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:43.695640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:43.695640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:43.699691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 413/834] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:43.700448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924cd059-f6c4-4599-a45a-e222e78dc0ed None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:43.702217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f54208-d908-462f-836d-a7ea39de83ad None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:43.747172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b40ce7-f593-4670-aefd-2a65ed00139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:43.748540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b40ce7-f593-4670-aefd-2a65ed00139a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.749046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b40ce7-f593-4670-aefd-2a65ed00139a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.749954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b40ce7-f593-4670-aefd-2a65ed00139a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:43.760671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b40ce7-f593-4670-aefd-2a65ed00139a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:43.763286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b40ce7-f593-4670-aefd-2a65ed00139a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-c4b40ce7-f593-4670-aefd-2a65ed00139a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:43.784092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:43.785094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:43.785985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:43.785985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:43.785985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:43.785985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:43.785985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:43.785985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:43.785985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:43.785985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:43.788989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 422/835] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:43.794289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:43.795737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b40ce7-f593-4670-aefd-2a65ed00139a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:43.828964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924cd059-f6c4-4599-a45a-e222e78dc0ed None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:43.866916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff526-7544-4d9f-afda-a97d8959431b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:43.867577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff526-7544-4d9f-afda-a97d8959431b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.867868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff526-7544-4d9f-afda-a97d8959431b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.868129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff526-7544-4d9f-afda-a97d8959431b None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:43.868845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff526-7544-4d9f-afda-a97d8959431b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:43.869151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff526-7544-4d9f-afda-a97d8959431b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:43.870505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff526-7544-4d9f-afda-a97d8959431b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:43.882799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 414/836] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:43.887304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228e1a2-4ba3-4a3c-8904-0474d6e934fd None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.887304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228e1a2-4ba3-4a3c-8904-0474d6e934fd None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.887304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228e1a2-4ba3-4a3c-8904-0474d6e934fd None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:43.887304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 415/837] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:43.890345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228e1a2-4ba3-4a3c-8904-0474d6e934fd None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:43.905610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b40ce7-f593-4670-aefd-2a65ed00139a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:43.914671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:43.944164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadbfd4-9838-4efa-8853-83ac508812ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:43.944756 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadbfd4-9838-4efa-8853-83ac508812ff None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.945037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadbfd4-9838-4efa-8853-83ac508812ff None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.945309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadbfd4-9838-4efa-8853-83ac508812ff None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:43.946089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbadbfd4-9838-4efa-8853-83ac508812ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:43.946395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbadbfd4-9838-4efa-8853-83ac508812ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:43.947856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbadbfd4-9838-4efa-8853-83ac508812ff None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:43.964800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 423/838] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:43.964800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9523e-ce07-4198-8dc6-cb8a786d111f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:43.964800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9523e-ce07-4198-8dc6-cb8a786d111f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:43.964800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9523e-ce07-4198-8dc6-cb8a786d111f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.004756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228e1a2-4ba3-4a3c-8904-0474d6e934fd None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:44.046112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4068d-2c40-47b7-9e21-10cc45124a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:44.046954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4068d-2c40-47b7-9e21-10cc45124a37 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.047246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4068d-2c40-47b7-9e21-10cc45124a37 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.047521 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4068d-2c40-47b7-9e21-10cc45124a37 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.051458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:44.057213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4068d-2c40-47b7-9e21-10cc45124a37 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:44.059243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4068d-2c40-47b7-9e21-10cc45124a37 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-f0b4068d-2c40-47b7-9e21-10cc45124a37 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:44.075344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:44.076337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 416/839] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:37:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:44.082262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4068d-2c40-47b7-9e21-10cc45124a37 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:44.097171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e1a9523e-ce07-4198-8dc6-cb8a786d111f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:44.098373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e1a9523e-ce07-4198-8dc6-cb8a786d111f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:44.171610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 424/840] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:44.172896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9523e-ce07-4198-8dc6-cb8a786d111f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:44.201772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4068d-2c40-47b7-9e21-10cc45124a37 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:44.239376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873399ce-40dd-4a92-ac6d-77fc5b7065b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:44.240145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873399ce-40dd-4a92-ac6d-77fc5b7065b4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.240553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873399ce-40dd-4a92-ac6d-77fc5b7065b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.240835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873399ce-40dd-4a92-ac6d-77fc5b7065b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.241674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873399ce-40dd-4a92-ac6d-77fc5b7065b4 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=4327e4eed5a1436ea1a00c2209bc5750) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:44.241991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873399ce-40dd-4a92-ac6d-77fc5b7065b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:44.243437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873399ce-40dd-4a92-ac6d-77fc5b7065b4 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:44.254764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 417/841] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:44 2022] GET /identity/v3/role_assignments?user.id=4327e4eed5a1436ea1a00c2209bc5750&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=433478cc318a467c931f13a757f161fa => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:44.254764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873399ce-40dd-4a92-ac6d-77fc5b7065b4 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:44.283815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:44.296835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9523e-ce07-4198-8dc6-cb8a786d111f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:44.334822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:44.339384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146be24d-58d2-4478-a8e0-861f83187142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:44.340119 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146be24d-58d2-4478-a8e0-861f83187142 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.340401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146be24d-58d2-4478-a8e0-861f83187142 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.340675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146be24d-58d2-4478-a8e0-861f83187142 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.341529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146be24d-58d2-4478-a8e0-861f83187142 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:44.341842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146be24d-58d2-4478-a8e0-861f83187142 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:44.343372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146be24d-58d2-4478-a8e0-861f83187142 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:44.372450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 425/842] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Jun 15 15:37:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:44.374760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146be24d-58d2-4478-a8e0-861f83187142 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:44.397810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873399ce-40dd-4a92-ac6d-77fc5b7065b4 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:44.436473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f42059-00b7-4de7-9833-447068989d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:44.437456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f42059-00b7-4de7-9833-447068989d6f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.438144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f42059-00b7-4de7-9833-447068989d6f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.438144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f42059-00b7-4de7-9833-447068989d6f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.438994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f42059-00b7-4de7-9833-447068989d6f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:44.441454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f42059-00b7-4de7-9833-447068989d6f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-34f42059-00b7-4de7-9833-447068989d6f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:44.459991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:44.461080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 15:37:44.461918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:44.462674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 418/843] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:44.468259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f42059-00b7-4de7-9833-447068989d6f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:44.515717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146be24d-58d2-4478-a8e0-861f83187142 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:44.561011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768e247-5766-4f32-a4f4-c961160cd614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:44.561998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768e247-5766-4f32-a4f4-c961160cd614 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.562455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768e247-5766-4f32-a4f4-c961160cd614 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.562928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768e247-5766-4f32-a4f4-c961160cd614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.588228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:44.602248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f42059-00b7-4de7-9833-447068989d6f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:44.640134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768e247-5766-4f32-a4f4-c961160cd614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:44.641143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:44.643751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768e247-5766-4f32-a4f4-c961160cd614 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:44.656544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9c81d-d5d1-4508-94fb-bcffa994f5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:44.657244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9c81d-d5d1-4508-94fb-bcffa994f5fe None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.657533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9c81d-d5d1-4508-94fb-bcffa994f5fe None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.657818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9c81d-d5d1-4508-94fb-bcffa994f5fe None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.658594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9c81d-d5d1-4508-94fb-bcffa994f5fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:44.659577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9c81d-d5d1-4508-94fb-bcffa994f5fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:44.662061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9c81d-d5d1-4508-94fb-bcffa994f5fe None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:44.669869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 419/844] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:44.674655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9c81d-d5d1-4508-94fb-bcffa994f5fe None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:44.763240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 426/845] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:44 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:37:44.764899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f187bd-dc09-4be2-8960-499ef5604023 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.765188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f187bd-dc09-4be2-8960-499ef5604023 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.765466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f187bd-dc09-4be2-8960-499ef5604023 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.766883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 427/846] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:44.772158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f187bd-dc09-4be2-8960-499ef5604023 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:44.788496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9c81d-d5d1-4508-94fb-bcffa994f5fe None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:44.822411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4493914-9529-420c-98cb-e3b742927269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:44.823101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4493914-9529-420c-98cb-e3b742927269 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4493914-9529-420c-98cb-e3b742927269 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.823701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4493914-9529-420c-98cb-e3b742927269 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.824346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4493914-9529-420c-98cb-e3b742927269 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:44.824650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4493914-9529-420c-98cb-e3b742927269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:44.825998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4493914-9529-420c-98cb-e3b742927269 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:44.830401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 420/847] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:44.831597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b492be-994a-48df-a82a-6aa231df29a2 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.831881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b492be-994a-48df-a82a-6aa231df29a2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.832152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b492be-994a-48df-a82a-6aa231df29a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.838134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:44.859705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-28b492be-994a-48df-a82a-6aa231df29a2 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:44.862165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-28b492be-994a-48df-a82a-6aa231df29a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:44.907548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f187bd-dc09-4be2-8960-499ef5604023 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:44.918015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:44.944319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 421/848] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:44.956075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124f224-e580-4071-8a49-1bfe3c088697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:44.956932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124f224-e580-4071-8a49-1bfe3c088697 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:44.957272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124f224-e580-4071-8a49-1bfe3c088697 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:44.957550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124f224-e580-4071-8a49-1bfe3c088697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:44.969885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124f224-e580-4071-8a49-1bfe3c088697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:44.971424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124f224-e580-4071-8a49-1bfe3c088697 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:44.971751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b492be-994a-48df-a82a-6aa231df29a2 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-6124f224-e580-4071-8a49-1bfe3c088697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:44.996728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:44.998487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:44.999593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:44.999593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:44.999593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:44.999593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:45.002893 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 428/849] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:45.005082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124f224-e580-4071-8a49-1bfe3c088697 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:45.108282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b492be-994a-48df-a82a-6aa231df29a2 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:45.125673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124f224-e580-4071-8a49-1bfe3c088697 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:45.139164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:45.145668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c1452-6eba-4836-befb-31fd3b0503b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:45.146765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c1452-6eba-4836-befb-31fd3b0503b1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:45.147231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c1452-6eba-4836-befb-31fd3b0503b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:45.147683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c1452-6eba-4836-befb-31fd3b0503b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:45.159218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c1452-6eba-4836-befb-31fd3b0503b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:45.167592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aefa1a-29d6-4252-aee7-d07fc0d24137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:45.168257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aefa1a-29d6-4252-aee7-d07fc0d24137 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:45.168535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aefa1a-29d6-4252-aee7-d07fc0d24137 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:45.168808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aefa1a-29d6-4252-aee7-d07fc0d24137 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-6f2c1452-6eba-4836-befb-31fd3b0503b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:45.169511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:45.170409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:37:45.171590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:45.172262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aefa1a-29d6-4252-aee7-d07fc0d24137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:45.172262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aefa1a-29d6-4252-aee7-d07fc0d24137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:45.172262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aefa1a-29d6-4252-aee7-d07fc0d24137 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:45.178883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 422/850] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:37:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:45.181092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c1452-6eba-4836-befb-31fd3b0503b1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:45.184321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 429/851] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:45.189184 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aefa1a-29d6-4252-aee7-d07fc0d24137 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:45.246117 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:45.301016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aefa1a-29d6-4252-aee7-d07fc0d24137 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:45.322972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c1452-6eba-4836-befb-31fd3b0503b1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:45.339207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129474e-7082-4368-9726-4329b039c317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:45.340351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129474e-7082-4368-9726-4329b039c317 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:45.340854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129474e-7082-4368-9726-4329b039c317 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:45.341327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129474e-7082-4368-9726-4329b039c317 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:45.348493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129474e-7082-4368-9726-4329b039c317 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:45.350690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129474e-7082-4368-9726-4329b039c317 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:45.363765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5af23-963e-4e10-95e2-c74a184b9119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:45.364414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5af23-963e-4e10-95e2-c74a184b9119 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:45.364699 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5af23-963e-4e10-95e2-c74a184b9119 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:45.364986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5af23-963e-4e10-95e2-c74a184b9119 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:45.365745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5af23-963e-4e10-95e2-c74a184b9119 None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:45.366061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5af23-963e-4e10-95e2-c74a184b9119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-e129474e-7082-4368-9726-4329b039c317 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:45.366666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:45.367571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:45.368483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:45.368483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:45.368483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:45.368483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:45.368483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:45.368483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:45.368483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:45.368483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:45.368927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5af23-963e-4e10-95e2-c74a184b9119 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:45.376682 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 430/852] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:45.377882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129474e-7082-4368-9726-4329b039c317 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:45.379715 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 423/853] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:37:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:45.387931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5af23-963e-4e10-95e2-c74a184b9119 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:45.415532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:45.507214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129474e-7082-4368-9726-4329b039c317 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:45.520822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5af23-963e-4e10-95e2-c74a184b9119 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:45.525571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:45.546947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd664f-eb58-463b-8145-b3fba81279ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:45.548000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd664f-eb58-463b-8145-b3fba81279ee None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:45.548496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd664f-eb58-463b-8145-b3fba81279ee None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:45.549073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd664f-eb58-463b-8145-b3fba81279ee None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:45.550409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd664f-eb58-463b-8145-b3fba81279ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:45.551106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd664f-eb58-463b-8145-b3fba81279ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:45.553525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd664f-eb58-463b-8145-b3fba81279ee None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:45.562647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24092c09-41d3-4090-8e77-7a4563fe0f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:45.563483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24092c09-41d3-4090-8e77-7a4563fe0f04 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:45.563771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24092c09-41d3-4090-8e77-7a4563fe0f04 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:45.564054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24092c09-41d3-4090-8e77-7a4563fe0f04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:45.573790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 431/854] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:45.575085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd664f-eb58-463b-8145-b3fba81279ee None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:45.576276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24092c09-41d3-4090-8e77-7a4563fe0f04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:45.577744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24092c09-41d3-4090-8e77-7a4563fe0f04 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-24092c09-41d3-4090-8e77-7a4563fe0f04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:45.601181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:45.602119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:45.603056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:45.603056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:45.603056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:45.603056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:45.606800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 424/855] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:45.608721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24092c09-41d3-4090-8e77-7a4563fe0f04 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:45.678965 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:45.697514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd664f-eb58-463b-8145-b3fba81279ee None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:45.741083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24092c09-41d3-4090-8e77-7a4563fe0f04 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:45.741853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e1654-63ef-4631-b367-9f1fb7ffa0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:45.742671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1654-63ef-4631-b367-9f1fb7ffa0f5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:45.743276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1654-63ef-4631-b367-9f1fb7ffa0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:45.743708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1654-63ef-4631-b367-9f1fb7ffa0f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:45.745424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e1654-63ef-4631-b367-9f1fb7ffa0f5 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=a6e3bb6553264387936e9a534afc3ac0) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:45.745890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e1654-63ef-4631-b367-9f1fb7ffa0f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:45.747501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e1654-63ef-4631-b367-9f1fb7ffa0f5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:45.755411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 432/856] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:45 2022] GET /identity/v3/role_assignments?user.id=a6e3bb6553264387936e9a534afc3ac0&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=433478cc318a467c931f13a757f161fa => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:45.779094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdaa8a-2416-4fff-aa8f-4c5b2a4d027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:45.780176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdaa8a-2416-4fff-aa8f-4c5b2a4d027b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:45.780662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdaa8a-2416-4fff-aa8f-4c5b2a4d027b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:45.781148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdaa8a-2416-4fff-aa8f-4c5b2a4d027b None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:45.782126 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdaa8a-2416-4fff-aa8f-4c5b2a4d027b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:45.782647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdaa8a-2416-4fff-aa8f-4c5b2a4d027b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:45.784299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdaa8a-2416-4fff-aa8f-4c5b2a4d027b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:45.798260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 425/857] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:45.804794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e1654-63ef-4631-b367-9f1fb7ffa0f5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:45.918446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e1654-63ef-4631-b367-9f1fb7ffa0f5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:45.957151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad2e65-3d66-4cba-8072-39a646e46b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:45.958413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad2e65-3d66-4cba-8072-39a646e46b2c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:45.958905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad2e65-3d66-4cba-8072-39a646e46b2c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:45.959179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad2e65-3d66-4cba-8072-39a646e46b2c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:45.968439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad2e65-3d66-4cba-8072-39a646e46b2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:45.969888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad2e65-3d66-4cba-8072-39a646e46b2c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-45ad2e65-3d66-4cba-8072-39a646e46b2c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:45.985167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:45.986138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:45.987162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 433/858] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:37:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:45.992791 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdaa8a-2416-4fff-aa8f-4c5b2a4d027b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:46.016418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:46.101949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdaa8a-2416-4fff-aa8f-4c5b2a4d027b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:46.137175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68d92-8180-4fe4-9ec3-8bd74fa3561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:46.137847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68d92-8180-4fe4-9ec3-8bd74fa3561b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:46.138128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68d92-8180-4fe4-9ec3-8bd74fa3561b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:46.138409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68d92-8180-4fe4-9ec3-8bd74fa3561b None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:46.139292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68d92-8180-4fe4-9ec3-8bd74fa3561b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:46.139608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68d92-8180-4fe4-9ec3-8bd74fa3561b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:46.141029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68d92-8180-4fe4-9ec3-8bd74fa3561b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:46.159476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 426/859] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Jun 15 15:37:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:46.164890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad2e65-3d66-4cba-8072-39a646e46b2c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:46.179182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:46.273224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad2e65-3d66-4cba-8072-39a646e46b2c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:46.307260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4dc24-7d86-4218-8ee1-67bcaf52bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:46.307908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4dc24-7d86-4218-8ee1-67bcaf52bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:46.308191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4dc24-7d86-4218-8ee1-67bcaf52bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:46.308465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4dc24-7d86-4218-8ee1-67bcaf52bdf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:46.377454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4dc24-7d86-4218-8ee1-67bcaf52bdf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:46.378997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4dc24-7d86-4218-8ee1-67bcaf52bdf8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:46.391158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:46.486302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 434/860] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:37:46.492885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68d92-8180-4fe4-9ec3-8bd74fa3561b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:46.534688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209edd73-e8fa-446c-8c67-a5dccd7ac194 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:46.535011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209edd73-e8fa-446c-8c67-a5dccd7ac194 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:46.535201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209edd73-e8fa-446c-8c67-a5dccd7ac194 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:46.536759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 435/861] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:46.541792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05314cd5-196a-4545-b4d1-a42140f5d15c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:46.542074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05314cd5-196a-4545-b4d1-a42140f5d15c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:46.542349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05314cd5-196a-4545-b4d1-a42140f5d15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:46.558909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-05314cd5-196a-4545-b4d1-a42140f5d15c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:46.560073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-05314cd5-196a-4545-b4d1-a42140f5d15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:46.582188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:46.611699 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68d92-8180-4fe4-9ec3-8bd74fa3561b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:46.643237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 436/862] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:46.652664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e19895-20db-4401-b7e7-dc6eff087ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:46.653913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e19895-20db-4401-b7e7-dc6eff087ed4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:46.654427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e19895-20db-4401-b7e7-dc6eff087ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:46.660751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e19895-20db-4401-b7e7-dc6eff087ed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:46.661859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05314cd5-196a-4545-b4d1-a42140f5d15c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:46.676411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e19895-20db-4401-b7e7-dc6eff087ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:46.678211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e19895-20db-4401-b7e7-dc6eff087ed4 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-87e19895-20db-4401-b7e7-dc6eff087ed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:46.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:46.702900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:46.704248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:46.704248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:46.704248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:46.704248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:46.704248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 427/863] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:46.711678 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e19895-20db-4401-b7e7-dc6eff087ed4 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:46.716291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:46.782895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05314cd5-196a-4545-b4d1-a42140f5d15c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:46.823137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f1dd2-c454-4c3d-8e61-dd469022ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:46.823866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f1dd2-c454-4c3d-8e61-dd469022ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:46.824143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f1dd2-c454-4c3d-8e61-dd469022ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:46.824429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f1dd2-c454-4c3d-8e61-dd469022ddf4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:46.825056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f1dd2-c454-4c3d-8e61-dd469022ddf4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:46.826420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f1dd2-c454-4c3d-8e61-dd469022ddf4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:46.829595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e19895-20db-4401-b7e7-dc6eff087ed4 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-192f1dd2-c454-4c3d-8e61-dd469022ddf4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:46.839481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:46.840442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 15:37:46.841289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:46.845521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 437/864] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Jun 15 15:37:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:46.846415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f1dd2-c454-4c3d-8e61-dd469022ddf4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:46.870385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:46.886133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95b903-9500-4f2f-a9a7-e80ce4699758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:46.886827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95b903-9500-4f2f-a9a7-e80ce4699758 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:46.887122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95b903-9500-4f2f-a9a7-e80ce4699758 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:46.887399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95b903-9500-4f2f-a9a7-e80ce4699758 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:46.888129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95b903-9500-4f2f-a9a7-e80ce4699758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:46.888431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95b903-9500-4f2f-a9a7-e80ce4699758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:46.889784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95b903-9500-4f2f-a9a7-e80ce4699758 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:46.908303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 428/865] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:46.910418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95b903-9500-4f2f-a9a7-e80ce4699758 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:46.975903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f1dd2-c454-4c3d-8e61-dd469022ddf4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:46.978986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:47.016935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bb226-13f7-4b30-8812-0e71d9266557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:47.017967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb226-13f7-4b30-8812-0e71d9266557 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:47.018420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95b903-9500-4f2f-a9a7-e80ce4699758 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:47.018676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb226-13f7-4b30-8812-0e71d9266557 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:47.019201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb226-13f7-4b30-8812-0e71d9266557 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:47.020393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bb226-13f7-4b30-8812-0e71d9266557 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:47.021442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bb226-13f7-4b30-8812-0e71d9266557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:47.022999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bb226-13f7-4b30-8812-0e71d9266557 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:47.027643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 438/866] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:37:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:47.029050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476618f8-b99b-4b95-8c82-23956c1d2034 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:47.029330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476618f8-b99b-4b95-8c82-23956c1d2034 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:47.029600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476618f8-b99b-4b95-8c82-23956c1d2034 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:47.030983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 439/867] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:47.035290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476618f8-b99b-4b95-8c82-23956c1d2034 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:47.059219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51535383-b116-4e68-8a7e-c75bfa6289ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:47.060095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51535383-b116-4e68-8a7e-c75bfa6289ba None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:47.060443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51535383-b116-4e68-8a7e-c75bfa6289ba None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:47.060728 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51535383-b116-4e68-8a7e-c75bfa6289ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:47.067790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51535383-b116-4e68-8a7e-c75bfa6289ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:47.069130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51535383-b116-4e68-8a7e-c75bfa6289ba None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-51535383-b116-4e68-8a7e-c75bfa6289ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:47.086373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:47.087574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:47.088866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:47.088866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:47.088866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:47.088866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:47.088866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:47.088866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:47.088866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:47.088866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:47.091248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 429/868] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:47.092896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe954a5-9ad5-4e26-b138-39a5ae30e163 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:47.093194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe954a5-9ad5-4e26-b138-39a5ae30e163 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:47.093477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe954a5-9ad5-4e26-b138-39a5ae30e163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:47.111893 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8fe954a5-9ad5-4e26-b138-39a5ae30e163 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:47.113695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8fe954a5-9ad5-4e26-b138-39a5ae30e163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:47.146968 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476618f8-b99b-4b95-8c82-23956c1d2034 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:47.159244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:47.182639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4989296-48a6-4f1c-bf7c-fbf67c3e2e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:47.183330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4989296-48a6-4f1c-bf7c-fbf67c3e2e35 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:47.183680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4989296-48a6-4f1c-bf7c-fbf67c3e2e35 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:47.183894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4989296-48a6-4f1c-bf7c-fbf67c3e2e35 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:47.184598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4989296-48a6-4f1c-bf7c-fbf67c3e2e35 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:47.184907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4989296-48a6-4f1c-bf7c-fbf67c3e2e35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:47.185663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 430/869] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:47.186295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe954a5-9ad5-4e26-b138-39a5ae30e163 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:47.186408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4989296-48a6-4f1c-bf7c-fbf67c3e2e35 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:47.198777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 440/870] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:37:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:47.209743 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4989296-48a6-4f1c-bf7c-fbf67c3e2e35 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:47.244571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:47.319453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe954a5-9ad5-4e26-b138-39a5ae30e163 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:47.327946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4989296-48a6-4f1c-bf7c-fbf67c3e2e35 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:47.357546 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e5a27-4d19-4c5c-8344-d5c8b8ad685e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:47.358748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e5a27-4d19-4c5c-8344-d5c8b8ad685e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:47.359246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e5a27-4d19-4c5c-8344-d5c8b8ad685e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:47.359745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e5a27-4d19-4c5c-8344-d5c8b8ad685e None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:47.360903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e5a27-4d19-4c5c-8344-d5c8b8ad685e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:47.361433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e5a27-4d19-4c5c-8344-d5c8b8ad685e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:47.363141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e5a27-4d19-4c5c-8344-d5c8b8ad685e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:47.365799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba2992-6e32-4a9c-bb35-98df333df075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:47.366585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba2992-6e32-4a9c-bb35-98df333df075 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:47.366903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba2992-6e32-4a9c-bb35-98df333df075 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:47.367182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba2992-6e32-4a9c-bb35-98df333df075 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:47.367916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba2992-6e32-4a9c-bb35-98df333df075 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:47.369299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba2992-6e32-4a9c-bb35-98df333df075 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:47.378080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 431/871] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:47.389079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e5a27-4d19-4c5c-8344-d5c8b8ad685e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:47.390868 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 441/872] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:37:47 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:37:47.437391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:47.521842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e5a27-4d19-4c5c-8344-d5c8b8ad685e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:47.568815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a859802-a6b8-4b15-9d2c-4b3c14792ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:47.569784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a859802-a6b8-4b15-9d2c-4b3c14792ec5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:47.570220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a859802-a6b8-4b15-9d2c-4b3c14792ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:47.570675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a859802-a6b8-4b15-9d2c-4b3c14792ec5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:47.571700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a859802-a6b8-4b15-9d2c-4b3c14792ec5 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=2f73ecf1501a4ab181ad9a24ac54aec8) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:47.572176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a859802-a6b8-4b15-9d2c-4b3c14792ec5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:47.573635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a859802-a6b8-4b15-9d2c-4b3c14792ec5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:47.583370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 432/873] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:47 2022] GET /identity/v3/role_assignments?user.id=2f73ecf1501a4ab181ad9a24ac54aec8&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:48.138078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990020ba-9857-4fa7-86b8-517854530eda None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:48.138629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990020ba-9857-4fa7-86b8-517854530eda None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:48.138629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990020ba-9857-4fa7-86b8-517854530eda None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:48.140259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 442/874] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:48.146300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5120d9-9450-4e77-a47d-490a49b30672 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:48.146588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5120d9-9450-4e77-a47d-490a49b30672 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:48.146900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5120d9-9450-4e77-a47d-490a49b30672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:48.164081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ff5120d9-9450-4e77-a47d-490a49b30672 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:48.165258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ff5120d9-9450-4e77-a47d-490a49b30672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:48.235939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 433/875] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:48 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:48.258091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990020ba-9857-4fa7-86b8-517854530eda None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:48.358685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:48.369850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990020ba-9857-4fa7-86b8-517854530eda None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:48.406582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438bd3c-7176-4e94-af45-806577bbcaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:48.407357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438bd3c-7176-4e94-af45-806577bbcaac None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:48.407639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438bd3c-7176-4e94-af45-806577bbcaac None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:48.407911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438bd3c-7176-4e94-af45-806577bbcaac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:48.408590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438bd3c-7176-4e94-af45-806577bbcaac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:48.410084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438bd3c-7176-4e94-af45-806577bbcaac None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-5438bd3c-7176-4e94-af45-806577bbcaac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:48.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:48.424089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:48.425333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 443/876] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:37:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:48.431815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5120d9-9450-4e77-a47d-490a49b30672 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:48.466421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:48.542039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5120d9-9450-4e77-a47d-490a49b30672 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:48.581510 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43462cf-3da6-4151-904a-87cddce4788c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:48.582183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43462cf-3da6-4151-904a-87cddce4788c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:48.582464 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43462cf-3da6-4151-904a-87cddce4788c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:48.583258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43462cf-3da6-4151-904a-87cddce4788c None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:48.583991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43462cf-3da6-4151-904a-87cddce4788c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:48.584531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43462cf-3da6-4151-904a-87cddce4788c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:48.585638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43462cf-3da6-4151-904a-87cddce4788c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:48.595039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 434/877] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:37:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:48.596074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438bd3c-7176-4e94-af45-806577bbcaac None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:48.705687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438bd3c-7176-4e94-af45-806577bbcaac None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:48.741622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9c17a-d19f-437c-aaa9-8f3c473faf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:48.742538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9c17a-d19f-437c-aaa9-8f3c473faf1f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:48.743001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9c17a-d19f-437c-aaa9-8f3c473faf1f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:48.743417 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9c17a-d19f-437c-aaa9-8f3c473faf1f None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:48.744257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9c17a-d19f-437c-aaa9-8f3c473faf1f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:48.744727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9c17a-d19f-437c-aaa9-8f3c473faf1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:48.746226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9c17a-d19f-437c-aaa9-8f3c473faf1f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:48.751478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 444/878] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:37:48 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:49.484707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4905a-a564-4e96-90d7-c82d1deec945 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:49.485151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4905a-a564-4e96-90d7-c82d1deec945 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:49.485559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4905a-a564-4e96-90d7-c82d1deec945 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:49.487305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 435/879] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:49.492544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c5ad-4681-4b09-83e7-bebc989863ff None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:49.493419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c5ad-4681-4b09-83e7-bebc989863ff None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:49.493910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c5ad-4681-4b09-83e7-bebc989863ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:49.511541 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6b26c5ad-4681-4b09-83e7-bebc989863ff None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:49.513032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6b26c5ad-4681-4b09-83e7-bebc989863ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:49.528725 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:49.591039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 445/880] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:49.613186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26c5ad-4681-4b09-83e7-bebc989863ff None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:49.613794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be83036-6838-4c10-9162-f9fdc4b9e9f0 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:49.614247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be83036-6838-4c10-9162-f9fdc4b9e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:49.614407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be83036-6838-4c10-9162-f9fdc4b9e9f0 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:49.616065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 436/881] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:49.621037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1bd2b-ada3-48e4-ae80-99c678d07605 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:49.621335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1bd2b-ada3-48e4-ae80-99c678d07605 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:49.621622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1bd2b-ada3-48e4-ae80-99c678d07605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:49.638494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-beb1bd2b-ada3-48e4-ae80-99c678d07605 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:49.639741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-beb1bd2b-ada3-48e4-ae80-99c678d07605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:49.715433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 437/882] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:49.729085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26c5ad-4681-4b09-83e7-bebc989863ff None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:49.738358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1bd2b-ada3-48e4-ae80-99c678d07605 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:49.760824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:49.774609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503d61-8c5b-4b22-bbd0-217f3589c7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:49.776332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503d61-8c5b-4b22-bbd0-217f3589c7e2 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:49.776793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503d61-8c5b-4b22-bbd0-217f3589c7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:49.777213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503d61-8c5b-4b22-bbd0-217f3589c7e2 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:49.778090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98503d61-8c5b-4b22-bbd0-217f3589c7e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:49.779640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98503d61-8c5b-4b22-bbd0-217f3589c7e2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:49.807465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 446/883] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:37:49 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:37:49.867621 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1bd2b-ada3-48e4-ae80-99c678d07605 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:49.872464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcf573-2ec2-4e8a-ac6b-9ccb957b7698 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:49.872753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcf573-2ec2-4e8a-ac6b-9ccb957b7698 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:49.873035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcf573-2ec2-4e8a-ac6b-9ccb957b7698 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:49.874681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 447/884] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:49.880350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf512c-db57-43b5-a59f-8dfe89c5a1d9 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:49.880631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf512c-db57-43b5-a59f-8dfe89c5a1d9 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:49.880917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf512c-db57-43b5-a59f-8dfe89c5a1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:49.904604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-38cf512c-db57-43b5-a59f-8dfe89c5a1d9 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:49.905869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-38cf512c-db57-43b5-a59f-8dfe89c5a1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:49.911613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:49.920342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e2388-56ba-441d-9547-139f03e0797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:49.921609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e2388-56ba-441d-9547-139f03e0797c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:49.922049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e2388-56ba-441d-9547-139f03e0797c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:49.922512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e2388-56ba-441d-9547-139f03e0797c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:49.923288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e2388-56ba-441d-9547-139f03e0797c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:49.924797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e2388-56ba-441d-9547-139f03e0797c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-235e2388-56ba-441d-9547-139f03e0797c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:49.937625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:49.938927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 15:37:49.939943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:49.944033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 438/885] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:49.945376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e2388-56ba-441d-9547-139f03e0797c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:50.005750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 448/886] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:50.030626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf512c-db57-43b5-a59f-8dfe89c5a1d9 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:50.039078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:50.062194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e2388-56ba-441d-9547-139f03e0797c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:50.100384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a76f9-0a58-4d58-93a6-80b1ae6ccb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:50.101042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a76f9-0a58-4d58-93a6-80b1ae6ccb6d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.101369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a76f9-0a58-4d58-93a6-80b1ae6ccb6d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.101604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a76f9-0a58-4d58-93a6-80b1ae6ccb6d None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.102291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a76f9-0a58-4d58-93a6-80b1ae6ccb6d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:50.102825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a76f9-0a58-4d58-93a6-80b1ae6ccb6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:50.104125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a76f9-0a58-4d58-93a6-80b1ae6ccb6d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:50.111044 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 439/887] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:50.117458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a76f9-0a58-4d58-93a6-80b1ae6ccb6d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:50.165465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf512c-db57-43b5-a59f-8dfe89c5a1d9 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:50.213971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38d1d6-ffbc-42f2-ae1e-f159cfdfc321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:50.215161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38d1d6-ffbc-42f2-ae1e-f159cfdfc321 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.216168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38d1d6-ffbc-42f2-ae1e-f159cfdfc321 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.216601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38d1d6-ffbc-42f2-ae1e-f159cfdfc321 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.228234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:50.233052 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38d1d6-ffbc-42f2-ae1e-f159cfdfc321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-3a38d1d6-ffbc-42f2-ae1e-f159cfdfc321 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:50.245292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:50.246431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:37:50.247571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:50.248240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a76f9-0a58-4d58-93a6-80b1ae6ccb6d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:50.251078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 449/888] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Jun 15 15:37:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:50.256334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38d1d6-ffbc-42f2-ae1e-f159cfdfc321 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:50.289523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bac76-39f1-4e15-97bf-0a3757742042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:50.290430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bac76-39f1-4e15-97bf-0a3757742042 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.290893 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bac76-39f1-4e15-97bf-0a3757742042 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.291317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bac76-39f1-4e15-97bf-0a3757742042 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.292188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bac76-39f1-4e15-97bf-0a3757742042 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:50.292643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bac76-39f1-4e15-97bf-0a3757742042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:50.294282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bac76-39f1-4e15-97bf-0a3757742042 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:50.299953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 440/889] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:50.306878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bac76-39f1-4e15-97bf-0a3757742042 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:50.318674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:50.368182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38d1d6-ffbc-42f2-ae1e-f159cfdfc321 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:50.405161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27afa0-b0ea-41a3-883e-bf194ff3c706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:50.405913 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27afa0-b0ea-41a3-883e-bf194ff3c706 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.406116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27afa0-b0ea-41a3-883e-bf194ff3c706 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.406413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27afa0-b0ea-41a3-883e-bf194ff3c706 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.407238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27afa0-b0ea-41a3-883e-bf194ff3c706 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:50.407557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27afa0-b0ea-41a3-883e-bf194ff3c706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:50.409174 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27afa0-b0ea-41a3-883e-bf194ff3c706 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:50.418080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bac76-39f1-4e15-97bf-0a3757742042 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:50.422221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 450/890] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Jun 15 15:37:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:50.423489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27afa0-b0ea-41a3-883e-bf194ff3c706 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:50.466207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0f9aa-3837-4dcd-a06c-69a1ceafee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:50.467112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f9aa-3837-4dcd-a06c-69a1ceafee8f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.467542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f9aa-3837-4dcd-a06c-69a1ceafee8f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.467992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0f9aa-3837-4dcd-a06c-69a1ceafee8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.468926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0f9aa-3837-4dcd-a06c-69a1ceafee8f None admin] RBAC: Adding query filter params (interface=public, service_id=0b8dd9bdb8774f7e8f92b33ee893a851, region_id=RegionOne) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:50.469386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0f9aa-3837-4dcd-a06c-69a1ceafee8f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:50.470914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0f9aa-3837-4dcd-a06c-69a1ceafee8f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:50.476720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 441/891] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Jun 15 15:37:50 2022] GET /identity/v3/endpoints?service_id=0b8dd9bdb8774f7e8f92b33ee893a851&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:50.485935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0f9aa-3837-4dcd-a06c-69a1ceafee8f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:50.491258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:50.532564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27afa0-b0ea-41a3-883e-bf194ff3c706 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:50.568337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31153b3a-8dac-4f55-b54a-417b555647f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:50.568875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:50.569134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31153b3a-8dac-4f55-b54a-417b555647f6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.569437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31153b3a-8dac-4f55-b54a-417b555647f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.569882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31153b3a-8dac-4f55-b54a-417b555647f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.581117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31153b3a-8dac-4f55-b54a-417b555647f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:50.582413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31153b3a-8dac-4f55-b54a-417b555647f6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-31153b3a-8dac-4f55-b54a-417b555647f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:50.600036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:50.601188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:50.602313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:50.602313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:50.602313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:50.602313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:50.605882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 451/892] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:50.607205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31153b3a-8dac-4f55-b54a-417b555647f6 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:50.610224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0f9aa-3837-4dcd-a06c-69a1ceafee8f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:50.648566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1ec54-804f-429d-add2-6e22241f77c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:50.649966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1ec54-804f-429d-add2-6e22241f77c5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.650436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1ec54-804f-429d-add2-6e22241f77c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.650890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1ec54-804f-429d-add2-6e22241f77c5 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.651858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1ec54-804f-429d-add2-6e22241f77c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:50.653492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1ec54-804f-429d-add2-6e22241f77c5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:50.667086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 442/893] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Jun 15 15:37:50 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:37:50.717407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31153b3a-8dac-4f55-b54a-417b555647f6 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:50.739726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:50.766754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6567ab1-2900-4465-aea7-919ec9163842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:50.767467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567ab1-2900-4465-aea7-919ec9163842 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.767750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567ab1-2900-4465-aea7-919ec9163842 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.768026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567ab1-2900-4465-aea7-919ec9163842 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.768753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6567ab1-2900-4465-aea7-919ec9163842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:50.769060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6567ab1-2900-4465-aea7-919ec9163842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:50.771316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6567ab1-2900-4465-aea7-919ec9163842 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:50.790950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 452/894] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:50.792415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1ec54-804f-429d-add2-6e22241f77c5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:50.915917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1ec54-804f-429d-add2-6e22241f77c5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:50.952464 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc5539-b6ea-41d6-a480-c0a55619a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:50.953557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc5539-b6ea-41d6-a480-c0a55619a298 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.953992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc5539-b6ea-41d6-a480-c0a55619a298 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.954416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc5539-b6ea-41d6-a480-c0a55619a298 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.954933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852caaad-5a8f-4596-bbe0-3abecb9baf71 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.955224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852caaad-5a8f-4596-bbe0-3abecb9baf71 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.955516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852caaad-5a8f-4596-bbe0-3abecb9baf71 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.957068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 453/895] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:50.958831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:50.962001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aac385-513d-411f-bb89-9bfaeb818d05 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:50.962277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aac385-513d-411f-bb89-9bfaeb818d05 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:50.962547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aac385-513d-411f-bb89-9bfaeb818d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:50.965149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc5539-b6ea-41d6-a480-c0a55619a298 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:50.966640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc5539-b6ea-41d6-a480-c0a55619a298 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:50.980415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-70aac385-513d-411f-bb89-9bfaeb818d05 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:50.981721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-70aac385-513d-411f-bb89-9bfaeb818d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-83cc5539-b6ea-41d6-a480-c0a55619a298 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:50.982939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:50.983869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:50.984950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 443/896] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:37:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:50.990476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc5539-b6ea-41d6-a480-c0a55619a298 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:51.080476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 454/897] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:51.102822 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aac385-513d-411f-bb89-9bfaeb818d05 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:51.105781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc5539-b6ea-41d6-a480-c0a55619a298 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:51.159707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c82626-9d70-493e-b3bc-61df33a92ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:51.161443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c82626-9d70-493e-b3bc-61df33a92ee5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:51.161921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c82626-9d70-493e-b3bc-61df33a92ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:51.162206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c82626-9d70-493e-b3bc-61df33a92ee5 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:51.163313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c82626-9d70-493e-b3bc-61df33a92ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:51.163643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c82626-9d70-493e-b3bc-61df33a92ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:51.165451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c82626-9d70-493e-b3bc-61df33a92ee5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:51.190883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 444/898] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Jun 15 15:37:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:51.194871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c82626-9d70-493e-b3bc-61df33a92ee5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:51.227239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aac385-513d-411f-bb89-9bfaeb818d05 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:51.229693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:51.251949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:51.267188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad506a-79e1-46a3-9931-6bc1d1921fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:51.268153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad506a-79e1-46a3-9931-6bc1d1921fd7 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:51.268628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad506a-79e1-46a3-9931-6bc1d1921fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:51.269376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad506a-79e1-46a3-9931-6bc1d1921fd7 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:51.270930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad506a-79e1-46a3-9931-6bc1d1921fd7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:51.272514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad506a-79e1-46a3-9931-6bc1d1921fd7 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:51.295568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 455/899] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:37:51 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:51.299593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c82626-9d70-493e-b3bc-61df33a92ee5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:51.345792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab21ebf-47ed-4d30-8e75-5ff46e37eaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:51.346373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab21ebf-47ed-4d30-8e75-5ff46e37eaf7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:51.346662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab21ebf-47ed-4d30-8e75-5ff46e37eaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:51.346977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab21ebf-47ed-4d30-8e75-5ff46e37eaf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:51.423479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab21ebf-47ed-4d30-8e75-5ff46e37eaf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:51.424960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab21ebf-47ed-4d30-8e75-5ff46e37eaf7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:51.480872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:51.534056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 445/900] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:37:51 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:37:51.540128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad506a-79e1-46a3-9931-6bc1d1921fd7 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:51.642850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad506a-79e1-46a3-9931-6bc1d1921fd7 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:51.679708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae1fb2-932b-4c14-8a9e-29dbd91e0de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:51.680514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae1fb2-932b-4c14-8a9e-29dbd91e0de8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:51.680798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae1fb2-932b-4c14-8a9e-29dbd91e0de8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:51.681078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae1fb2-932b-4c14-8a9e-29dbd91e0de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:51.692018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae1fb2-932b-4c14-8a9e-29dbd91e0de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:51.693989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae1fb2-932b-4c14-8a9e-29dbd91e0de8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-a9ae1fb2-932b-4c14-8a9e-29dbd91e0de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:51.711426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:51.712347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:51.713286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:51.713286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:51.713286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:51.713286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:51.713286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 456/901] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:51.719595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab21ebf-47ed-4d30-8e75-5ff46e37eaf7 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:51.831090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab21ebf-47ed-4d30-8e75-5ff46e37eaf7 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:51.870712 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013e0d4-6c15-4c41-a64f-b4c9e2d3b4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:51.871801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013e0d4-6c15-4c41-a64f-b4c9e2d3b4b1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:51.872311 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013e0d4-6c15-4c41-a64f-b4c9e2d3b4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:51.872808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013e0d4-6c15-4c41-a64f-b4c9e2d3b4b1 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:51.873756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013e0d4-6c15-4c41-a64f-b4c9e2d3b4b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:51.874271 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013e0d4-6c15-4c41-a64f-b4c9e2d3b4b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:51.875902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013e0d4-6c15-4c41-a64f-b4c9e2d3b4b1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:51.889623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 446/902] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:51.895778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae1fb2-932b-4c14-8a9e-29dbd91e0de8 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:51.915555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:52.005316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae1fb2-932b-4c14-8a9e-29dbd91e0de8 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:52.042072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf582b-ac74-4ea6-a3d5-41541025d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:52.042886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf582b-ac74-4ea6-a3d5-41541025d561 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:52.043217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf582b-ac74-4ea6-a3d5-41541025d561 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:52.043494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf582b-ac74-4ea6-a3d5-41541025d561 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:52.051301 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf582b-ac74-4ea6-a3d5-41541025d561 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:52.053352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf582b-ac74-4ea6-a3d5-41541025d561 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-57bf582b-ac74-4ea6-a3d5-41541025d561 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:52.069264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:52.070291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:52.071322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:52.071322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:52.071322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:52.071322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:52.071322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:52.071322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:52.071322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:52.071322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:52.071322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 457/903] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:52.078320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013e0d4-6c15-4c41-a64f-b4c9e2d3b4b1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:52.092657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:52.126344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46132c-8284-4b7f-812d-48baf8e4b747 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:52.126673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46132c-8284-4b7f-812d-48baf8e4b747 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:52.126944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46132c-8284-4b7f-812d-48baf8e4b747 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:52.128431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 458/904] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:52.134075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66312b7a-6979-4fb8-8184-1fe35f7ef5fb None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:52.134611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66312b7a-6979-4fb8-8184-1fe35f7ef5fb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:52.135202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66312b7a-6979-4fb8-8184-1fe35f7ef5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:52.152612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-66312b7a-6979-4fb8-8184-1fe35f7ef5fb None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:52.154121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-66312b7a-6979-4fb8-8184-1fe35f7ef5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:52.190569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013e0d4-6c15-4c41-a64f-b4c9e2d3b4b1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:52.225780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 459/905] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:52.229317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebb5e8-63b6-4b3c-b682-8114c08b782d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:52.230230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebb5e8-63b6-4b3c-b682-8114c08b782d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:52.230784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebb5e8-63b6-4b3c-b682-8114c08b782d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:52.231282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebb5e8-63b6-4b3c-b682-8114c08b782d None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:52.232277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebb5e8-63b6-4b3c-b682-8114c08b782d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:52.232798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebb5e8-63b6-4b3c-b682-8114c08b782d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:52.234427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebb5e8-63b6-4b3c-b682-8114c08b782d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:52.247645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 447/906] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:52.253341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66312b7a-6979-4fb8-8184-1fe35f7ef5fb None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:52.255281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebb5e8-63b6-4b3c-b682-8114c08b782d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:52.264063 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:52.328767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:52.360268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebb5e8-63b6-4b3c-b682-8114c08b782d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:52.366027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66312b7a-6979-4fb8-8184-1fe35f7ef5fb None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:52.396455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f234e96-0362-4a82-adb2-9115f552c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:52.397529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f234e96-0362-4a82-adb2-9115f552c2ec None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:52.398063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f234e96-0362-4a82-adb2-9115f552c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:52.398478 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f234e96-0362-4a82-adb2-9115f552c2ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:52.399399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f234e96-0362-4a82-adb2-9115f552c2ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:52.400897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f234e96-0362-4a82-adb2-9115f552c2ec None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:52.403851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2441e7-e3ae-491e-bb3d-8d1d62db1ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:52.404451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2441e7-e3ae-491e-bb3d-8d1d62db1ab9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:52.404735 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2441e7-e3ae-491e-bb3d-8d1d62db1ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:52.405069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2441e7-e3ae-491e-bb3d-8d1d62db1ab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:52.405828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2441e7-e3ae-491e-bb3d-8d1d62db1ab9 None admin] RBAC: Adding query filter params (role.id=7a222a4a0d3b41b08325b7ad1e9b6322, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=2f73ecf1501a4ab181ad9a24ac54aec8) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:52.406138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2441e7-e3ae-491e-bb3d-8d1d62db1ab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:52.407588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2441e7-e3ae-491e-bb3d-8d1d62db1ab9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:52.418768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 460/907] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:37:52 2022] GET /identity/v3/role_assignments?user.id=2f73ecf1501a4ab181ad9a24ac54aec8&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=7a222a4a0d3b41b08325b7ad1e9b6322 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-9f234e96-0362-4a82-adb2-9115f552c2ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:52.418987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:52.419964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:52.420827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 448/908] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Jun 15 15:37:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:52.428888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2441e7-e3ae-491e-bb3d-8d1d62db1ab9 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:52.527101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:52.559708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2441e7-e3ae-491e-bb3d-8d1d62db1ab9 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:52.596215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2231772-83dc-45d3-ac96-c6be3c189e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:52.596885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2231772-83dc-45d3-ac96-c6be3c189e5f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:52.597168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2231772-83dc-45d3-ac96-c6be3c189e5f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:52.597445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2231772-83dc-45d3-ac96-c6be3c189e5f None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:52.598685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2231772-83dc-45d3-ac96-c6be3c189e5f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:52.599023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2231772-83dc-45d3-ac96-c6be3c189e5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:52.600387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2231772-83dc-45d3-ac96-c6be3c189e5f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:52.605503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 461/909] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:37:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:52.611059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f234e96-0362-4a82-adb2-9115f552c2ec None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:52.717343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f234e96-0362-4a82-adb2-9115f552c2ec None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:52.762627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502cf65-b0de-49d7-814a-25887d050ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:52.764176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502cf65-b0de-49d7-814a-25887d050ddb None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:52.764692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502cf65-b0de-49d7-814a-25887d050ddb None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:52.765214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502cf65-b0de-49d7-814a-25887d050ddb None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:52.766122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502cf65-b0de-49d7-814a-25887d050ddb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:52.766632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502cf65-b0de-49d7-814a-25887d050ddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:52.768215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502cf65-b0de-49d7-814a-25887d050ddb None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:52.771027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:52.774576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 449/910] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:37:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:52.780939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2231772-83dc-45d3-ac96-c6be3c189e5f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:52.898345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2231772-83dc-45d3-ac96-c6be3c189e5f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:52.933812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223adf2-97bf-4702-b22d-3d89a46c93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:52.934974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223adf2-97bf-4702-b22d-3d89a46c93e8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:52.935473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223adf2-97bf-4702-b22d-3d89a46c93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:52.935958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223adf2-97bf-4702-b22d-3d89a46c93e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:52.936952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223adf2-97bf-4702-b22d-3d89a46c93e8 None admin] RBAC: Adding query filter params (interface=public, service_id=03422f5395c44b9f88b9166b40dca5a5, region_id=RegionOne) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:52.937473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223adf2-97bf-4702-b22d-3d89a46c93e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:52.939141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223adf2-97bf-4702-b22d-3d89a46c93e8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:52.940911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:52.949727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 462/911] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Jun 15 15:37:52 2022] GET /identity/v3/endpoints?service_id=03422f5395c44b9f88b9166b40dca5a5&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:52.951173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502cf65-b0de-49d7-814a-25887d050ddb None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:53.050149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6b51b-fc8d-4eb3-a2f2-2fde8e9be268 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.050450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6b51b-fc8d-4eb3-a2f2-2fde8e9be268 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.050675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6b51b-fc8d-4eb3-a2f2-2fde8e9be268 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.052380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 463/912] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:53.057181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3193c0b-830e-470e-9d4b-76f655103b1e None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.057469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3193c0b-830e-470e-9d4b-76f655103b1e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.057856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3193c0b-830e-470e-9d4b-76f655103b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.064614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502cf65-b0de-49d7-814a-25887d050ddb None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:53.074521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-f3193c0b-830e-470e-9d4b-76f655103b1e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:53.075746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f3193c0b-830e-470e-9d4b-76f655103b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:53.100485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd90293-d891-40e2-b91b-21f762de7b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:53.101298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd90293-d891-40e2-b91b-21f762de7b77 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.101682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd90293-d891-40e2-b91b-21f762de7b77 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.102068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd90293-d891-40e2-b91b-21f762de7b77 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.102897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd90293-d891-40e2-b91b-21f762de7b77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:53.104345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd90293-d891-40e2-b91b-21f762de7b77 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:53.115803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 450/913] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Jun 15 15:37:52 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:37:53.163156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 464/914] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:53.187270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd90293-d891-40e2-b91b-21f762de7b77 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:53.257812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:53.300345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd90293-d891-40e2-b91b-21f762de7b77 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:53.335258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652dd5fa-846f-41ab-80fb-9ac4175ea15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:53.336058 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652dd5fa-846f-41ab-80fb-9ac4175ea15b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.336340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652dd5fa-846f-41ab-80fb-9ac4175ea15b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.336617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652dd5fa-846f-41ab-80fb-9ac4175ea15b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.337270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652dd5fa-846f-41ab-80fb-9ac4175ea15b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:53.338688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652dd5fa-846f-41ab-80fb-9ac4175ea15b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-652dd5fa-846f-41ab-80fb-9ac4175ea15b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:53.351222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:53.352213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:53.353050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 451/915] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:53.360300 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3193c0b-830e-470e-9d4b-76f655103b1e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:53.474372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3193c0b-830e-470e-9d4b-76f655103b1e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:53.507156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:53.513631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58fe92-7c7e-4471-8e6f-c4e2a8aeb070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:53.514320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58fe92-7c7e-4471-8e6f-c4e2a8aeb070 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.514906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58fe92-7c7e-4471-8e6f-c4e2a8aeb070 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.515017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58fe92-7c7e-4471-8e6f-c4e2a8aeb070 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.515657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58fe92-7c7e-4471-8e6f-c4e2a8aeb070 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:53.516060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58fe92-7c7e-4471-8e6f-c4e2a8aeb070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:53.517794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58fe92-7c7e-4471-8e6f-c4e2a8aeb070 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:53.525265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 465/916] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:53.530860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652dd5fa-846f-41ab-80fb-9ac4175ea15b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:53.565250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aac2d-050d-4c00-b2d9-d84148924445 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.565752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aac2d-050d-4c00-b2d9-d84148924445 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.565752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aac2d-050d-4c00-b2d9-d84148924445 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.568014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 466/917] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:53.574829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2cb71-9131-46ed-960f-9f436bd6fd5f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.574829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2cb71-9131-46ed-960f-9f436bd6fd5f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.574829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2cb71-9131-46ed-960f-9f436bd6fd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.591854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-21b2cb71-9131-46ed-960f-9f436bd6fd5f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:53.593107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-21b2cb71-9131-46ed-960f-9f436bd6fd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:53.644273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652dd5fa-846f-41ab-80fb-9ac4175ea15b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:53.671877 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:53.682396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8350f-0c3d-4644-9a6d-f4c1666a54dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:53.683436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8350f-0c3d-4644-9a6d-f4c1666a54dd None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.683865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8350f-0c3d-4644-9a6d-f4c1666a54dd None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.684235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8350f-0c3d-4644-9a6d-f4c1666a54dd None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.685064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8350f-0c3d-4644-9a6d-f4c1666a54dd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:53.685348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 467/918] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:53.690891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8350f-0c3d-4644-9a6d-f4c1666a54dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:53.692888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8350f-0c3d-4644-9a6d-f4c1666a54dd None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:53.698714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 452/919] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Jun 15 15:37:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:53.707880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2cb71-9131-46ed-960f-9f436bd6fd5f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:53.709258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8350f-0c3d-4644-9a6d-f4c1666a54dd None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:53.824588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2cb71-9131-46ed-960f-9f436bd6fd5f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:53.825456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8350f-0c3d-4644-9a6d-f4c1666a54dd None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:53.850977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:53.864879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82899153-ba3b-4071-bbad-ccddf2f3306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:53.865993 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82899153-ba3b-4071-bbad-ccddf2f3306d None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.866597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82899153-ba3b-4071-bbad-ccddf2f3306d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.867199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82899153-ba3b-4071-bbad-ccddf2f3306d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.868391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82899153-ba3b-4071-bbad-ccddf2f3306d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:53.868917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ab3d8-67ae-439a-9e0a-620c0253a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:53.869716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab3d8-67ae-439a-9e0a-620c0253a49d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:53.869991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab3d8-67ae-439a-9e0a-620c0253a49d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:53.870261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab3d8-67ae-439a-9e0a-620c0253a49d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:53.870919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82899153-ba3b-4071-bbad-ccddf2f3306d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:53.871046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ab3d8-67ae-439a-9e0a-620c0253a49d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:53.872255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ab3d8-67ae-439a-9e0a-620c0253a49d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-489ab3d8-67ae-439a-9e0a-620c0253a49d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:53.887572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:53.888665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:37:53.889496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:53.892651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 453/920] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:37:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:53.895795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ab3d8-67ae-439a-9e0a-620c0253a49d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:53.897580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 468/921] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Jun 15 15:37:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:53.959194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:54.010062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ab3d8-67ae-439a-9e0a-620c0253a49d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:54.053778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b88d7-edb9-4c36-ae8a-ab82eaec8108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:54.054456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b88d7-edb9-4c36-ae8a-ab82eaec8108 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:54.054750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b88d7-edb9-4c36-ae8a-ab82eaec8108 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:54.055034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b88d7-edb9-4c36-ae8a-ab82eaec8108 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:54.055747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b88d7-edb9-4c36-ae8a-ab82eaec8108 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:54.056049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b88d7-edb9-4c36-ae8a-ab82eaec8108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:54.057422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b88d7-edb9-4c36-ae8a-ab82eaec8108 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:54.063096 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 454/922] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:37:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:54.069389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82899153-ba3b-4071-bbad-ccddf2f3306d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:54.177702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82899153-ba3b-4071-bbad-ccddf2f3306d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:54.214242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b12135-fd4f-4be8-a117-cdb3093d9bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:54.215302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b12135-fd4f-4be8-a117-cdb3093d9bbd None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:54.215790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b12135-fd4f-4be8-a117-cdb3093d9bbd None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:54.216278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b12135-fd4f-4be8-a117-cdb3093d9bbd None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:54.217202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b12135-fd4f-4be8-a117-cdb3093d9bbd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:54.217716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b12135-fd4f-4be8-a117-cdb3093d9bbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:54.219330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b12135-fd4f-4be8-a117-cdb3093d9bbd None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:54.225754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 469/923] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:37:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:54.231273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b88d7-edb9-4c36-ae8a-ab82eaec8108 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:54.337181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b88d7-edb9-4c36-ae8a-ab82eaec8108 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:54.351889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:54.378749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a06b4-4c83-4e2c-8a5e-7b75555a0995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:54.379796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a06b4-4c83-4e2c-8a5e-7b75555a0995 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:54.380279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a06b4-4c83-4e2c-8a5e-7b75555a0995 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:54.380762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a06b4-4c83-4e2c-8a5e-7b75555a0995 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:54.381976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a06b4-4c83-4e2c-8a5e-7b75555a0995 None admin] RBAC: Adding query filter params (interface=public, service_id=a3ab5a32ad53487a9f7f99c625f2c36a, region_id=RegionOne) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:54.382498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a06b4-4c83-4e2c-8a5e-7b75555a0995 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:54.384084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a06b4-4c83-4e2c-8a5e-7b75555a0995 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:54.393407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 455/924] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Jun 15 15:37:54 2022] GET /identity/v3/endpoints?service_id=a3ab5a32ad53487a9f7f99c625f2c36a&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:54.399955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b12135-fd4f-4be8-a117-cdb3093d9bbd None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:54.451869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:54.518554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b12135-fd4f-4be8-a117-cdb3093d9bbd None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:54.554435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fae3fe-0cb3-4f15-b651-b797927cb118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:54.555137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fae3fe-0cb3-4f15-b651-b797927cb118 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:54.555473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fae3fe-0cb3-4f15-b651-b797927cb118 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:54.555758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fae3fe-0cb3-4f15-b651-b797927cb118 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:54.556457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fae3fe-0cb3-4f15-b651-b797927cb118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:54.557864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fae3fe-0cb3-4f15-b651-b797927cb118 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:54.565188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 470/925] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Jun 15 15:37:54 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:37:54.635943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dbf89-9e97-4809-b057-ab249750ec59 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:54.636603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dbf89-9e97-4809-b057-ab249750ec59 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:54.637145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dbf89-9e97-4809-b057-ab249750ec59 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:54.639040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 456/926] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:54.644773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34ca4a-ebb0-45ec-af74-819ded0a3f93 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:54.645805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34ca4a-ebb0-45ec-af74-819ded0a3f93 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:54.645896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34ca4a-ebb0-45ec-af74-819ded0a3f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:54.664592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-8e34ca4a-ebb0-45ec-af74-819ded0a3f93 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:54.665798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8e34ca4a-ebb0-45ec-af74-819ded0a3f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:54.754886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 471/927] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:54.774697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dbf89-9e97-4809-b057-ab249750ec59 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:54.880696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dbf89-9e97-4809-b057-ab249750ec59 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:54.922889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f32f1-3a43-4164-b538-586120dc3107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:54.922889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f32f1-3a43-4164-b538-586120dc3107 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:54.922889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f32f1-3a43-4164-b538-586120dc3107 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:54.922889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f32f1-3a43-4164-b538-586120dc3107 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:54.932757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f32f1-3a43-4164-b538-586120dc3107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-ee7f32f1-3a43-4164-b538-586120dc3107 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:54.941294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:54.943149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:37:54.944150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:54.946413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 457/928] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Jun 15 15:37:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:54.952318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34ca4a-ebb0-45ec-af74-819ded0a3f93 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:54.989364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:55.062490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34ca4a-ebb0-45ec-af74-819ded0a3f93 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:55.098503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049ce70-e42c-4687-a3c0-d3c3f5ca92f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:55.099459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049ce70-e42c-4687-a3c0-d3c3f5ca92f6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.099905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049ce70-e42c-4687-a3c0-d3c3f5ca92f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.100339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049ce70-e42c-4687-a3c0-d3c3f5ca92f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.101230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049ce70-e42c-4687-a3c0-d3c3f5ca92f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:55.101701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049ce70-e42c-4687-a3c0-d3c3f5ca92f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:55.103259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049ce70-e42c-4687-a3c0-d3c3f5ca92f6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:55.112223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 472/929] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Jun 15 15:37:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:55.118152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f32f1-3a43-4164-b538-586120dc3107 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:55.162062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:55.226665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f32f1-3a43-4164-b538-586120dc3107 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:55.262762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2507e0-cce6-49f1-8d06-0e36d29bad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:55.263604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2507e0-cce6-49f1-8d06-0e36d29bad13 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.263891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2507e0-cce6-49f1-8d06-0e36d29bad13 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.264168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2507e0-cce6-49f1-8d06-0e36d29bad13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.276814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2507e0-cce6-49f1-8d06-0e36d29bad13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:55.278303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2507e0-cce6-49f1-8d06-0e36d29bad13 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-3f2507e0-cce6-49f1-8d06-0e36d29bad13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:55.298797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:55.299804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:55.301673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:55.301673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:55.301673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:55.301673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:55.301673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 458/930] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:55.307820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049ce70-e42c-4687-a3c0-d3c3f5ca92f6 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:55.333981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6bf20-9e93-468f-85b1-ed78e08dcb66 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.334169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6bf20-9e93-468f-85b1-ed78e08dcb66 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.334500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6bf20-9e93-468f-85b1-ed78e08dcb66 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.337000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 459/931] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:55.341852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c5be1-dc46-4922-b20a-c7629e8225e6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.342139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c5be1-dc46-4922-b20a-c7629e8225e6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.342419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c5be1-dc46-4922-b20a-c7629e8225e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.359891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-883c5be1-dc46-4922-b20a-c7629e8225e6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:55.361172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-883c5be1-dc46-4922-b20a-c7629e8225e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:55.414014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:55.418031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049ce70-e42c-4687-a3c0-d3c3f5ca92f6 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:55.442970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 460/932] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:37:55.453661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c18dc-4059-4f24-8dcf-a1545baf30de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:55.454327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c18dc-4059-4f24-8dcf-a1545baf30de None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.454614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c18dc-4059-4f24-8dcf-a1545baf30de None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.454925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c18dc-4059-4f24-8dcf-a1545baf30de None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.455648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c18dc-4059-4f24-8dcf-a1545baf30de None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:55.455961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c18dc-4059-4f24-8dcf-a1545baf30de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:55.457426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c18dc-4059-4f24-8dcf-a1545baf30de None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:55.459728 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c5be1-dc46-4922-b20a-c7629e8225e6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:55.470235 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 473/933] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:55.476233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c18dc-4059-4f24-8dcf-a1545baf30de None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:55.481629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:55.572327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c5be1-dc46-4922-b20a-c7629e8225e6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:55.603015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c18dc-4059-4f24-8dcf-a1545baf30de None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:55.608328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66860619-e4ca-40fa-88ab-258b4496a91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:55.609223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66860619-e4ca-40fa-88ab-258b4496a91a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.609577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66860619-e4ca-40fa-88ab-258b4496a91a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.609927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66860619-e4ca-40fa-88ab-258b4496a91a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.610655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66860619-e4ca-40fa-88ab-258b4496a91a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:55.612142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66860619-e4ca-40fa-88ab-258b4496a91a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-66860619-e4ca-40fa-88ab-258b4496a91a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:55.625651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:55.626593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 15:37:55.627473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:55.632002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 461/934] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Jun 15 15:37:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:55.632924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66860619-e4ca-40fa-88ab-258b4496a91a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:55.649760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec3d40-8b7c-48d1-b494-3f1845f9ce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:55.651926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec3d40-8b7c-48d1-b494-3f1845f9ce38 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.652381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec3d40-8b7c-48d1-b494-3f1845f9ce38 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.652815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec3d40-8b7c-48d1-b494-3f1845f9ce38 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.663767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec3d40-8b7c-48d1-b494-3f1845f9ce38 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:55.666090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec3d40-8b7c-48d1-b494-3f1845f9ce38 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-79ec3d40-8b7c-48d1-b494-3f1845f9ce38 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:37:55.687031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:55.688334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:37:55.689523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:55.696988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 474/935] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:37:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:55.698824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec3d40-8b7c-48d1-b494-3f1845f9ce38 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:55.718498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:55.749195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:55.771899 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66860619-e4ca-40fa-88ab-258b4496a91a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:55.813003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046e9ef-70e8-4e2a-ab33-f2a4fbc08acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:55.813680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e9ef-70e8-4e2a-ab33-f2a4fbc08acf None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.813968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e9ef-70e8-4e2a-ab33-f2a4fbc08acf None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.814249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e9ef-70e8-4e2a-ab33-f2a4fbc08acf None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.815229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046e9ef-70e8-4e2a-ab33-f2a4fbc08acf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:55.815546 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046e9ef-70e8-4e2a-ab33-f2a4fbc08acf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:55.817542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046e9ef-70e8-4e2a-ab33-f2a4fbc08acf None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:55.819794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec3d40-8b7c-48d1-b494-3f1845f9ce38 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:55.827065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 462/936] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:37:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:55.831813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046e9ef-70e8-4e2a-ab33-f2a4fbc08acf None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:55.861098 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac294430-8a65-43c1-bee9-63c78f43d2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:55.861709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac294430-8a65-43c1-bee9-63c78f43d2e0 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.861994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac294430-8a65-43c1-bee9-63c78f43d2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.862272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac294430-8a65-43c1-bee9-63c78f43d2e0 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.863146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac294430-8a65-43c1-bee9-63c78f43d2e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:55.863483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac294430-8a65-43c1-bee9-63c78f43d2e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:55.865038 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac294430-8a65-43c1-bee9-63c78f43d2e0 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:55.888119 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 475/937] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Jun 15 15:37:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:55.890285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac294430-8a65-43c1-bee9-63c78f43d2e0 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:55.948649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046e9ef-70e8-4e2a-ab33-f2a4fbc08acf None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:55.986987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02ff69-b322-44c0-aa35-857e996f94a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:55.987985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02ff69-b322-44c0-aa35-857e996f94a9 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:55.988473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02ff69-b322-44c0-aa35-857e996f94a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:55.989045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02ff69-b322-44c0-aa35-857e996f94a9 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:55.990009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02ff69-b322-44c0-aa35-857e996f94a9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:55.990537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02ff69-b322-44c0-aa35-857e996f94a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:55.991243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:55.993704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02ff69-b322-44c0-aa35-857e996f94a9 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:56.004989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 463/938] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:37:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:56.006629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02ff69-b322-44c0-aa35-857e996f94a9 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:56.021685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac294430-8a65-43c1-bee9-63c78f43d2e0 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:56.023985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:56.069269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4e101-5560-4979-adc2-089203e71edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:56.070480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4e101-5560-4979-adc2-089203e71edb None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.070818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4e101-5560-4979-adc2-089203e71edb None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.071109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4e101-5560-4979-adc2-089203e71edb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.083446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4e101-5560-4979-adc2-089203e71edb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:56.084819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4e101-5560-4979-adc2-089203e71edb None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-9be4e101-5560-4979-adc2-089203e71edb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:56.104058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:56.104998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:56.111159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:56.111159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:56.111159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:56.111159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:56.111159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 476/939] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:56.113010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4e101-5560-4979-adc2-089203e71edb None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:56.126213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02ff69-b322-44c0-aa35-857e996f94a9 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:56.171087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0dd8e-deb6-4113-98af-11242533ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:56.171778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0dd8e-deb6-4113-98af-11242533ecf8 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.172067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0dd8e-deb6-4113-98af-11242533ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.172348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0dd8e-deb6-4113-98af-11242533ecf8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.173689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0dd8e-deb6-4113-98af-11242533ecf8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:56.175172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0dd8e-deb6-4113-98af-11242533ecf8 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:56.206065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 464/940] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Jun 15 15:37:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:56.231570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ee739-87ea-420f-adac-e92af013faa8 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.231887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ee739-87ea-420f-adac-e92af013faa8 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.232124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ee739-87ea-420f-adac-e92af013faa8 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.238125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 465/941] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:56.240164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a62d4d-760c-4740-bd3a-d30e0ad1b383 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.240453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a62d4d-760c-4740-bd3a-d30e0ad1b383 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.240729 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a62d4d-760c-4740-bd3a-d30e0ad1b383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.248341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4e101-5560-4979-adc2-089203e71edb None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:56.258589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-38a62d4d-760c-4740-bd3a-d30e0ad1b383 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:56.259903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-38a62d4d-760c-4740-bd3a-d30e0ad1b383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:56.283005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:56.300360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d88c72-06d6-49ba-831d-d6515d8eb47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:56.301677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d88c72-06d6-49ba-831d-d6515d8eb47a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.302044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d88c72-06d6-49ba-831d-d6515d8eb47a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.302386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d88c72-06d6-49ba-831d-d6515d8eb47a None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.303210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d88c72-06d6-49ba-831d-d6515d8eb47a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:56.303582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d88c72-06d6-49ba-831d-d6515d8eb47a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:56.305064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d88c72-06d6-49ba-831d-d6515d8eb47a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:56.321234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 477/942] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:56.328497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d88c72-06d6-49ba-831d-d6515d8eb47a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:56.341863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:56.359087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 466/943] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:56.377967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a62d4d-760c-4740-bd3a-d30e0ad1b383 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:56.450429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d88c72-06d6-49ba-831d-d6515d8eb47a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:56.494236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4d432-28b7-430a-94ed-6e888f17e778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:56.495263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4d432-28b7-430a-94ed-6e888f17e778 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.495552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4d432-28b7-430a-94ed-6e888f17e778 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.495838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4d432-28b7-430a-94ed-6e888f17e778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.504939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4d432-28b7-430a-94ed-6e888f17e778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:56.506907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4d432-28b7-430a-94ed-6e888f17e778 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:56.515768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a62d4d-760c-4740-bd3a-d30e0ad1b383 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-0af4d432-28b7-430a-94ed-6e888f17e778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:56.524233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:56.525109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:56.525995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:56.525995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:56.525995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:56.525995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:56.525995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:56.525995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:56.525995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:56.525995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:56.528640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 478/944] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:56.538770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4d432-28b7-430a-94ed-6e888f17e778 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:56.560420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:56.561256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.561546 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.561840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.563181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:56.565691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:56.568271 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING keystone.server.flask.common [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 15:37:56.587833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.sql.core [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 15:37:56.587833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 15:37:56.587833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [parameters: {'id': '835536fa968543eea566354e2680ada7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 15:37:56.587833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86639) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 15:37:56.590478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 835536fa968543eea566354e2680ada7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 835536fa968543eea566354e2680ada7. Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application conn.query(q) Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 15:37:56.599429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result.read() Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 15 15:37:56.600265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application next(self.gen) Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application yield resource Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application next(self.gen) Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise self.value Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application session.commit() Jun 15 15:37:56.601253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.session.flush() Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application compat.raise_( Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 15:37:56.602135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 15:37:56.603083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application conn.query(q) Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result.read() Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 15:37:56.603886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [parameters: {'id': '835536fa968543eea566354e2680ada7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:56.604745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:56.605519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:56.606445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:56.606445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 15 15:37:56.606445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 15 15:37:56.606445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:56.606445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:56.606445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 15 15:37:56.606445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 15 15:37:56.606445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 835536fa968543eea566354e2680ada7. Jun 15 15:37:56.606445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:56.608847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 467/945] 10.222.0.62 () {60 vars in 1140 bytes} [Wed Jun 15 15:37:56 2022] POST /identity/v3/projects => generated 262 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:56.622689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:56.654305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4d432-28b7-430a-94ed-6e888f17e778 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:56.693083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad58a-5079-460a-ba9e-35f60bbbb893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:56.694153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad58a-5079-460a-ba9e-35f60bbbb893 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.694443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad58a-5079-460a-ba9e-35f60bbbb893 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.694721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad58a-5079-460a-ba9e-35f60bbbb893 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.695852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad58a-5079-460a-ba9e-35f60bbbb893 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:56.696165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad58a-5079-460a-ba9e-35f60bbbb893 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:56.697791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad58a-5079-460a-ba9e-35f60bbbb893 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:56.704724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:56.717827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 479/946] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:56.719255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad58a-5079-460a-ba9e-35f60bbbb893 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:56.777828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e37d74-82a5-4aa1-958b-b1d90be28bc1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:56.813765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12730e9-66a1-4afc-946e-7059846f1ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:56.814564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12730e9-66a1-4afc-946e-7059846f1ba1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.814883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12730e9-66a1-4afc-946e-7059846f1ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.815160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12730e9-66a1-4afc-946e-7059846f1ba1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.823425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12730e9-66a1-4afc-946e-7059846f1ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:56.825187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12730e9-66a1-4afc-946e-7059846f1ba1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-a12730e9-66a1-4afc-946e-7059846f1ba1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:37:56.839771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:56.840675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:56.841566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:37:56.841566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:37:56.841566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:37:56.841566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:37:56.841566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:37:56.841566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:37:56.841566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:37:56.841566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:56.841566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 468/947] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:56.842525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f83ee-3ab4-4c43-befa-4b026f47e71e None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.842632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f83ee-3ab4-4c43-befa-4b026f47e71e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.842942 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f83ee-3ab4-4c43-befa-4b026f47e71e None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.850370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 469/948] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:56.850370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f83ee-3ab4-4c43-befa-4b026f47e71e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:56.859141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad58a-5079-460a-ba9e-35f60bbbb893 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:56.862551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:56.896325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6913a22-bb16-44bc-aa60-5da7abb524f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:56.897249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6913a22-bb16-44bc-aa60-5da7abb524f2 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.897529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6913a22-bb16-44bc-aa60-5da7abb524f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.897811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6913a22-bb16-44bc-aa60-5da7abb524f2 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/2f73ecf1501a4ab181ad9a24ac54aec8/roles/7a222a4a0d3b41b08325b7ad1e9b6322` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.900110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6913a22-bb16-44bc-aa60-5da7abb524f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=2f73ecf1501a4ab181ad9a24ac54aec8, role_id=7a222a4a0d3b41b08325b7ad1e9b6322)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:56.901558 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6913a22-bb16-44bc-aa60-5da7abb524f2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:56.915202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 480/949] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:37:56 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/2f73ecf1501a4ab181ad9a24ac54aec8/roles/7a222a4a0d3b41b08325b7ad1e9b6322 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:37:56.929983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e9867-06e5-4302-9e11-258f1c60a289 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:56.930361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e9867-06e5-4302-9e11-258f1c60a289 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:56.930719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e9867-06e5-4302-9e11-258f1c60a289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:56.954320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-a25e9867-06e5-4302-9e11-258f1c60a289 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:56.955971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a25e9867-06e5-4302-9e11-258f1c60a289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:56.980244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f83ee-3ab4-4c43-befa-4b026f47e71e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:57.001726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:57.037493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4c7f2-fda4-4be8-9540-73d06bd0d056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:57.038456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4c7f2-fda4-4be8-9540-73d06bd0d056 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:57.038763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4c7f2-fda4-4be8-9540-73d06bd0d056 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:57.039045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4c7f2-fda4-4be8-9540-73d06bd0d056 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:57.040363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4c7f2-fda4-4be8-9540-73d06bd0d056 None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:57.040681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4c7f2-fda4-4be8-9540-73d06bd0d056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:57.042424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4c7f2-fda4-4be8-9540-73d06bd0d056 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:57.052960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 481/950] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:57.067488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 470/951] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:37:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:57.082967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e9867-06e5-4302-9e11-258f1c60a289 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:57.196360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:57.203891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e9867-06e5-4302-9e11-258f1c60a289 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:57.242921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ebfc0-9c13-45bf-98f8-bfca5a8aa509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:57.243729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ebfc0-9c13-45bf-98f8-bfca5a8aa509 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:57.244009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ebfc0-9c13-45bf-98f8-bfca5a8aa509 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:57.244284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ebfc0-9c13-45bf-98f8-bfca5a8aa509 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:57.244948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ebfc0-9c13-45bf-98f8-bfca5a8aa509 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:57.246314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ebfc0-9c13-45bf-98f8-bfca5a8aa509 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-8f2ebfc0-9c13-45bf-98f8-bfca5a8aa509 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:57.258578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:57.259687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:37:57.260531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:57.264753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 482/952] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:37:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:57.266165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4c7f2-fda4-4be8-9540-73d06bd0d056 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:57.376602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4c7f2-fda4-4be8-9540-73d06bd0d056 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:57.413953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e418f6-1134-41fa-9b0c-3db1adf7e435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:57.414623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e418f6-1134-41fa-9b0c-3db1adf7e435 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:57.414950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e418f6-1134-41fa-9b0c-3db1adf7e435 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:57.415226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e418f6-1134-41fa-9b0c-3db1adf7e435 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:57.415940 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e418f6-1134-41fa-9b0c-3db1adf7e435 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:57.416245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e418f6-1134-41fa-9b0c-3db1adf7e435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:57.417616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e418f6-1134-41fa-9b0c-3db1adf7e435 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:57.426851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 471/953] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:37:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:57.429652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ebfc0-9c13-45bf-98f8-bfca5a8aa509 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:57.540767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ebfc0-9c13-45bf-98f8-bfca5a8aa509 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:57.579552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa40aba-82ed-4f36-a6fe-eb5aa5011bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:57.580222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa40aba-82ed-4f36-a6fe-eb5aa5011bbf None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:57.580503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa40aba-82ed-4f36-a6fe-eb5aa5011bbf None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:57.580779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa40aba-82ed-4f36-a6fe-eb5aa5011bbf None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:57.581491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa40aba-82ed-4f36-a6fe-eb5aa5011bbf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:57.581797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa40aba-82ed-4f36-a6fe-eb5aa5011bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:57.583729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa40aba-82ed-4f36-a6fe-eb5aa5011bbf None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:57.592596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 483/954] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:37:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:37:57.594753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e418f6-1134-41fa-9b0c-3db1adf7e435 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:57.637073 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:57.706705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e418f6-1134-41fa-9b0c-3db1adf7e435 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:57.743551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8e3de-597c-492c-ae08-e28243d1de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:57.744569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8e3de-597c-492c-ae08-e28243d1de9d None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:57.745060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8e3de-597c-492c-ae08-e28243d1de9d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:57.745543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8e3de-597c-492c-ae08-e28243d1de9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:57.746535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8e3de-597c-492c-ae08-e28243d1de9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:57.748133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8e3de-597c-492c-ae08-e28243d1de9d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:57.768342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 472/955] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Jun 15 15:37:57 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:37:58.485773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403148e-933c-4d4b-ab80-2d2052c60b4b None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:58.486319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403148e-933c-4d4b-ab80-2d2052c60b4b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:58.486319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403148e-933c-4d4b-ab80-2d2052c60b4b None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:58.488002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 484/956] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:58.493508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4348d-1a90-4e2f-a7f8-75bcf566d301 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:58.493968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4348d-1a90-4e2f-a7f8-75bcf566d301 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:58.494265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4348d-1a90-4e2f-a7f8-75bcf566d301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:58.512105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-a6b4348d-1a90-4e2f-a7f8-75bcf566d301 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:58.513748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a6b4348d-1a90-4e2f-a7f8-75bcf566d301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:58.590703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 473/957] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:58.612841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403148e-933c-4d4b-ab80-2d2052c60b4b None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:58.638672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:58.719105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403148e-933c-4d4b-ab80-2d2052c60b4b None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:58.752849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1397cf9-e2d5-479d-9f04-d22f257bb2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:58.753636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1397cf9-e2d5-479d-9f04-d22f257bb2f5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:58.754433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1397cf9-e2d5-479d-9f04-d22f257bb2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:58.754783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1397cf9-e2d5-479d-9f04-d22f257bb2f5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:58.755438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1397cf9-e2d5-479d-9f04-d22f257bb2f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:58.756862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1397cf9-e2d5-479d-9f04-d22f257bb2f5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-c1397cf9-e2d5-479d-9f04-d22f257bb2f5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:58.769165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:58.770853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 15:37:58.771765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:58.778861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 485/958] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:37:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:58.781108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4348d-1a90-4e2f-a7f8-75bcf566d301 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:58.808080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:58.891267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4348d-1a90-4e2f-a7f8-75bcf566d301 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:58.926885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad5cce-733f-441c-8a20-85c9ce7091d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:58.927552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad5cce-733f-441c-8a20-85c9ce7091d5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:58.927829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad5cce-733f-441c-8a20-85c9ce7091d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:58.928100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad5cce-733f-441c-8a20-85c9ce7091d5 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:58.928801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad5cce-733f-441c-8a20-85c9ce7091d5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:58.929105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad5cce-733f-441c-8a20-85c9ce7091d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:58.930453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad5cce-733f-441c-8a20-85c9ce7091d5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:58.940001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 474/959] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:37:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:58.941377 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1397cf9-e2d5-479d-9f04-d22f257bb2f5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:59.032956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:59.048293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1397cf9-e2d5-479d-9f04-d22f257bb2f5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:59.082701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd0d88-8236-46d4-be12-d0daf3ab90b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:59.083478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd0d88-8236-46d4-be12-d0daf3ab90b2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.083761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd0d88-8236-46d4-be12-d0daf3ab90b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.084037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd0d88-8236-46d4-be12-d0daf3ab90b2 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.084807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd0d88-8236-46d4-be12-d0daf3ab90b2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:59.085118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd0d88-8236-46d4-be12-d0daf3ab90b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:59.086532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd0d88-8236-46d4-be12-d0daf3ab90b2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:59.098709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 486/960] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:37:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:59.168537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e4aed-79f8-4fa3-8124-f741abd50f46 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.168864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e4aed-79f8-4fa3-8124-f741abd50f46 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.169095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e4aed-79f8-4fa3-8124-f741abd50f46 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.170668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 475/961] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:37:59.176276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba657b7f-2556-4b6c-8a9a-cc3e0a24321f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.176549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba657b7f-2556-4b6c-8a9a-cc3e0a24321f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.176827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba657b7f-2556-4b6c-8a9a-cc3e0a24321f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.195748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ba657b7f-2556-4b6c-8a9a-cc3e0a24321f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:59.196950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ba657b7f-2556-4b6c-8a9a-cc3e0a24321f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:59.274910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 487/962] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:59.300670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e4aed-79f8-4fa3-8124-f741abd50f46 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:59.389543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:59.408053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e4aed-79f8-4fa3-8124-f741abd50f46 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:59.447778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512e13f-59d5-4d58-afb4-6a68456a0ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:59.448600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512e13f-59d5-4d58-afb4-6a68456a0ba8 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.448881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512e13f-59d5-4d58-afb4-6a68456a0ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.449158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512e13f-59d5-4d58-afb4-6a68456a0ba8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.460743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512e13f-59d5-4d58-afb4-6a68456a0ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-7512e13f-59d5-4d58-afb4-6a68456a0ba8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:37:59.468998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:59.469924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:59.470812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 476/963] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Jun 15 15:37:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:59.476187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba657b7f-2556-4b6c-8a9a-cc3e0a24321f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:59.491237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acdcbc-b57a-4ec9-9cff-ad4c2b9a3e1f None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.491562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acdcbc-b57a-4ec9-9cff-ad4c2b9a3e1f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.491823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acdcbc-b57a-4ec9-9cff-ad4c2b9a3e1f None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.494205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 477/964] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:37:59.499347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dffcbb-bd18-4ad7-9371-9da3f0deb5db None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.499635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dffcbb-bd18-4ad7-9371-9da3f0deb5db None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.499917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dffcbb-bd18-4ad7-9371-9da3f0deb5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.517182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c2dffcbb-bd18-4ad7-9371-9da3f0deb5db None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:37:59.518361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c2dffcbb-bd18-4ad7-9371-9da3f0deb5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:37:59.563439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:59.586018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba657b7f-2556-4b6c-8a9a-cc3e0a24321f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:59.594899 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 478/965] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:37:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:37:59.616264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dffcbb-bd18-4ad7-9371-9da3f0deb5db None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:59.623661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37eaf03-dc85-43b6-bd61-7ef405ad7a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:59.624516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37eaf03-dc85-43b6-bd61-7ef405ad7a33 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.624807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37eaf03-dc85-43b6-bd61-7ef405ad7a33 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.625088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37eaf03-dc85-43b6-bd61-7ef405ad7a33 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.625838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37eaf03-dc85-43b6-bd61-7ef405ad7a33 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:37:59.626297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37eaf03-dc85-43b6-bd61-7ef405ad7a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:59.627917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37eaf03-dc85-43b6-bd61-7ef405ad7a33 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:59.638709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 488/966] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Jun 15 15:37:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:37:59.645223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37eaf03-dc85-43b6-bd61-7ef405ad7a33 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:59.701928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:59.729615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dffcbb-bd18-4ad7-9371-9da3f0deb5db None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:59.759100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37eaf03-dc85-43b6-bd61-7ef405ad7a33 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:59.765657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:59.766413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.767210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.767569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.768346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:59.770497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:59.787003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.sql.core [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 15:37:59.787003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 15:37:59.787003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [parameters: {'user_id': 'b272d7ed0d7d4be89fdd4808fe06245d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 15:37:59.787003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86639) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 15:37:59.795788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0572c-f8fb-41a4-bbf1-29d06de331ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application conn.query(q) Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result.read() Jun 15 15:37:59.796364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 15:37:59.797802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application next(self.gen) Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application yield resource Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application next(self.gen) Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise self.value Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application session.commit() Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 15:37:59.799958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.session.flush() Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application compat.raise_( Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 15:37:59.802128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 15:37:59.803700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application conn.query(q) Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application result.read() Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 15:37:59.817074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b272d7ed0d7d4be89fdd4808fe06245d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:59.818149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 15 15:37:59.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 15 15:37:59.820453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 15 15:37:59.820453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 15 15:37:59.820453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 15:37:59.820453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 15:37:59.820453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 15:37:59.820453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:37:59.820876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0572c-f8fb-41a4-bbf1-29d06de331ec None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.820876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0572c-f8fb-41a4-bbf1-29d06de331ec None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.820876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0572c-f8fb-41a4-bbf1-29d06de331ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.821127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 479/967] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Jun 15 15:37:59 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:37:59.821127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:59.821279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0572c-f8fb-41a4-bbf1-29d06de331ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:37:59.821279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0572c-f8fb-41a4-bbf1-29d06de331ec None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:37:59.821453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-69b0572c-f8fb-41a4-bbf1-29d06de331ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:37:59.841248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:37:59.842918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:37:59.844737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:37:59.844737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:37:59.844737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:37:59.844737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:37:59.850347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 489/968] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:37:59.851391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0572c-f8fb-41a4-bbf1-29d06de331ec None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:37:59.942380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c2b8e-0f7d-4013-80c7-a7662daa84b5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:59.971379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0572c-f8fb-41a4-bbf1-29d06de331ec None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:37:59.987125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cd1c3-3bac-4346-978e-48a33a3776f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:37:59.988206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cd1c3-3bac-4346-978e-48a33a3776f0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:37:59.988499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cd1c3-3bac-4346-978e-48a33a3776f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:37:59.989175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cd1c3-3bac-4346-978e-48a33a3776f0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:37:59.993123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:37:59.998898 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cd1c3-3bac-4346-978e-48a33a3776f0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.001525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cd1c3-3bac-4346-978e-48a33a3776f0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.009161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c04a4-b50a-43da-88cc-e37726a77799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:00.010306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c04a4-b50a-43da-88cc-e37726a77799 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.010841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c04a4-b50a-43da-88cc-e37726a77799 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.011381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c04a4-b50a-43da-88cc-e37726a77799 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.012106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c04a4-b50a-43da-88cc-e37726a77799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:00.012412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c04a4-b50a-43da-88cc-e37726a77799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.013764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c04a4-b50a-43da-88cc-e37726a77799 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-c83cd1c3-3bac-4346-978e-48a33a3776f0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:38:00.022340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:00.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 15:38:00.024305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:00.027587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 480/969] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:37:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:00.029563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b0c02-38f1-4710-9346-86561d342f6f None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.029866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b0c02-38f1-4710-9346-86561d342f6f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.030149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b0c02-38f1-4710-9346-86561d342f6f None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.031796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 490/970] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:00.031796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c04a4-b50a-43da-88cc-e37726a77799 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:00.034593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 481/971] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:00.039210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b0c02-38f1-4710-9346-86561d342f6f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:00.100492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:00.147449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c04a4-b50a-43da-88cc-e37726a77799 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:00.149633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b0c02-38f1-4710-9346-86561d342f6f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:00.181826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2f3f8-dfa4-45fe-a896-84092101aaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:00.182499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2f3f8-dfa4-45fe-a896-84092101aaf2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.182810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2f3f8-dfa4-45fe-a896-84092101aaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.183094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2f3f8-dfa4-45fe-a896-84092101aaf2 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.183887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2f3f8-dfa4-45fe-a896-84092101aaf2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:00.184204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2f3f8-dfa4-45fe-a896-84092101aaf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.185643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2f3f8-dfa4-45fe-a896-84092101aaf2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.188308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46cb54-68f9-442b-99e8-64d3a5078c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:00.189315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46cb54-68f9-442b-99e8-64d3a5078c75 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.189794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46cb54-68f9-442b-99e8-64d3a5078c75 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.190249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46cb54-68f9-442b-99e8-64d3a5078c75 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.198884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46cb54-68f9-442b-99e8-64d3a5078c75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.200554 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46cb54-68f9-442b-99e8-64d3a5078c75 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-ed46cb54-68f9-442b-99e8-64d3a5078c75 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:38:00.216431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:00.217394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 15:38:00.218556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:00.220877 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 482/972] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:00.223104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 491/973] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:00.226688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2f3f8-dfa4-45fe-a896-84092101aaf2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:00.232146 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558335-7b05-4ee3-9a90-ca0a56a61675 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.235196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558335-7b05-4ee3-9a90-ca0a56a61675 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.235746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24558335-7b05-4ee3-9a90-ca0a56a61675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.261633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-24558335-7b05-4ee3-9a90-ca0a56a61675 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:00.264590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-24558335-7b05-4ee3-9a90-ca0a56a61675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:00.329321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:00.355712 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 483/974] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:00.366978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2f3f8-dfa4-45fe-a896-84092101aaf2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:00.377963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558335-7b05-4ee3-9a90-ca0a56a61675 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:00.382018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:00.408059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873df18-f7a9-4bf4-b590-611a0d79582d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:00.409038 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873df18-f7a9-4bf4-b590-611a0d79582d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.409475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873df18-f7a9-4bf4-b590-611a0d79582d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.409906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873df18-f7a9-4bf4-b590-611a0d79582d None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.411517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873df18-f7a9-4bf4-b590-611a0d79582d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:00.411835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873df18-f7a9-4bf4-b590-611a0d79582d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.413273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873df18-f7a9-4bf4-b590-611a0d79582d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.439477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 492/975] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:00.444753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873df18-f7a9-4bf4-b590-611a0d79582d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:00.515710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24558335-7b05-4ee3-9a90-ca0a56a61675 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:00.555473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628ed6d-239c-4e75-a634-37714c8ea88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:00.556647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628ed6d-239c-4e75-a634-37714c8ea88c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.556927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628ed6d-239c-4e75-a634-37714c8ea88c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.557196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628ed6d-239c-4e75-a634-37714c8ea88c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.557841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628ed6d-239c-4e75-a634-37714c8ea88c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.559710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628ed6d-239c-4e75-a634-37714c8ea88c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.570805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873df18-f7a9-4bf4-b590-611a0d79582d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-5628ed6d-239c-4e75-a634-37714c8ea88c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:00.575321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:00.576396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:00.583105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 484/976] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:00.584798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628ed6d-239c-4e75-a634-37714c8ea88c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:00.608222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff0820-1bd1-40d8-83a9-23b66f56dc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:00.608803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0820-1bd1-40d8-83a9-23b66f56dc87 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.609083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0820-1bd1-40d8-83a9-23b66f56dc87 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.609746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0820-1bd1-40d8-83a9-23b66f56dc87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.639197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:00.652588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:00.685702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff0820-1bd1-40d8-83a9-23b66f56dc87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.687303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff0820-1bd1-40d8-83a9-23b66f56dc87 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.692912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628ed6d-239c-4e75-a634-37714c8ea88c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:00.728551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8471e0f-c7be-4af7-a582-20b6fe9a2880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:00.729217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8471e0f-c7be-4af7-a582-20b6fe9a2880 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.729499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8471e0f-c7be-4af7-a582-20b6fe9a2880 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.729815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8471e0f-c7be-4af7-a582-20b6fe9a2880 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.730530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8471e0f-c7be-4af7-a582-20b6fe9a2880 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:00.730870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8471e0f-c7be-4af7-a582-20b6fe9a2880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.732227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8471e0f-c7be-4af7-a582-20b6fe9a2880 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.737124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 485/977] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:00.742516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8471e0f-c7be-4af7-a582-20b6fe9a2880 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:00.799906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 493/978] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 15:38:00.805696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff0820-1bd1-40d8-83a9-23b66f56dc87 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:00.849957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8471e0f-c7be-4af7-a582-20b6fe9a2880 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:00.866255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:00.883301 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:00.898214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366813e5-e3a3-47d1-abb1-f32e95ad6937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:00.899137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366813e5-e3a3-47d1-abb1-f32e95ad6937 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.899727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366813e5-e3a3-47d1-abb1-f32e95ad6937 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.900232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366813e5-e3a3-47d1-abb1-f32e95ad6937 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.901208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366813e5-e3a3-47d1-abb1-f32e95ad6937 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:00.901682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366813e5-e3a3-47d1-abb1-f32e95ad6937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.903245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366813e5-e3a3-47d1-abb1-f32e95ad6937 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.908474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 486/979] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:00.914169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366813e5-e3a3-47d1-abb1-f32e95ad6937 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:00.921250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff0820-1bd1-40d8-83a9-23b66f56dc87 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:00.959966 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c360c-ee04-4bf8-9822-ab0e47a92f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:00.960762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c360c-ee04-4bf8-9822-ab0e47a92f4f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:00.961041 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c360c-ee04-4bf8-9822-ab0e47a92f4f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:00.961314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c360c-ee04-4bf8-9822-ab0e47a92f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:00.972336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c360c-ee04-4bf8-9822-ab0e47a92f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:00.973786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c360c-ee04-4bf8-9822-ab0e47a92f4f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-374c360c-ee04-4bf8-9822-ab0e47a92f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:38:00.991686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:00.992644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:00.994102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:38:00.994102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:38:00.994102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:00.994102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:00.994102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 494/980] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:00.998277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c360c-ee04-4bf8-9822-ab0e47a92f4f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:01.027578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366813e5-e3a3-47d1-abb1-f32e95ad6937 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:01.064468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c5e97-0d76-47a6-9c80-f3c99c80a516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:01.065390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c5e97-0d76-47a6-9c80-f3c99c80a516 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:01.065674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c5e97-0d76-47a6-9c80-f3c99c80a516 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:01.065951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c5e97-0d76-47a6-9c80-f3c99c80a516 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:01.067368 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c5e97-0d76-47a6-9c80-f3c99c80a516 None admin] RBAC: Adding query filter params (interface=admin, service_id=a3ab5a32ad53487a9f7f99c625f2c36a, region_id=RegionOne) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:01.067685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c5e97-0d76-47a6-9c80-f3c99c80a516 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:01.069048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c5e97-0d76-47a6-9c80-f3c99c80a516 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:01.074324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 487/981] 10.222.0.62 () {56 vars in 1249 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/endpoints?service_id=a3ab5a32ad53487a9f7f99c625f2c36a&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:01.079657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c5e97-0d76-47a6-9c80-f3c99c80a516 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:01.105271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c360c-ee04-4bf8-9822-ab0e47a92f4f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:01.130691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:01.143122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b6010-9eb8-41c1-ac60-459e95f9536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:01.144163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b6010-9eb8-41c1-ac60-459e95f9536a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:01.144486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b6010-9eb8-41c1-ac60-459e95f9536a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:01.144766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b6010-9eb8-41c1-ac60-459e95f9536a None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:01.145483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b6010-9eb8-41c1-ac60-459e95f9536a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:01.145898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b6010-9eb8-41c1-ac60-459e95f9536a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:01.147324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b6010-9eb8-41c1-ac60-459e95f9536a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:01.151529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:01.164818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 495/982] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:01.165824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b6010-9eb8-41c1-ac60-459e95f9536a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:01.219130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c5e97-0d76-47a6-9c80-f3c99c80a516 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:01.255037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232771b7-06e2-4b08-9262-ebad571d3a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:01.255705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232771b7-06e2-4b08-9262-ebad571d3a9d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:01.255987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232771b7-06e2-4b08-9262-ebad571d3a9d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:01.256263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232771b7-06e2-4b08-9262-ebad571d3a9d None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:01.256945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232771b7-06e2-4b08-9262-ebad571d3a9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:01.258299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232771b7-06e2-4b08-9262-ebad571d3a9d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:01.264419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 488/983] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Jun 15 15:38:01 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:38:01.278647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b6010-9eb8-41c1-ac60-459e95f9536a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:01.318236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8003f11-ecc7-4a40-a823-c3990ee39fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:01.319356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8003f11-ecc7-4a40-a823-c3990ee39fbd None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:01.319788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8003f11-ecc7-4a40-a823-c3990ee39fbd None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:01.320238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8003f11-ecc7-4a40-a823-c3990ee39fbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:01.328858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8003f11-ecc7-4a40-a823-c3990ee39fbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:01.330632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8003f11-ecc7-4a40-a823-c3990ee39fbd None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-b8003f11-ecc7-4a40-a823-c3990ee39fbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:38:01.347161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:01.348296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:01.349209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:38:01.349209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:38:01.349209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:38:01.349209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:38:01.349209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:38:01.349209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:38:01.349209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:38:01.349209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:01.350917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 496/984] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:01.355426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232771b7-06e2-4b08-9262-ebad571d3a9d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:01.404591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f523e-217f-40fa-a0aa-e12477019cb8 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:01.404953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f523e-217f-40fa-a0aa-e12477019cb8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:01.405374 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f523e-217f-40fa-a0aa-e12477019cb8 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:01.407109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 497/985] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:38:01.412709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab92eaf-f828-45a5-b78d-4302edadb9de None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:01.413023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab92eaf-f828-45a5-b78d-4302edadb9de None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:01.413274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab92eaf-f828-45a5-b78d-4302edadb9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:01.431637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-fab92eaf-f828-45a5-b78d-4302edadb9de None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:01.432828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fab92eaf-f828-45a5-b78d-4302edadb9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:01.477400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232771b7-06e2-4b08-9262-ebad571d3a9d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:01.479752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:01.485841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:01.512295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58fd78-d2fd-4d61-a443-153712c49707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:01.512950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58fd78-d2fd-4d61-a443-153712c49707 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:01.513231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58fd78-d2fd-4d61-a443-153712c49707 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:01.513504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58fd78-d2fd-4d61-a443-153712c49707 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:01.514274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58fd78-d2fd-4d61-a443-153712c49707 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:01.514580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58fd78-d2fd-4d61-a443-153712c49707 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:01.516016 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58fd78-d2fd-4d61-a443-153712c49707 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:01.519044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 498/986] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:38:01.530764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 489/987] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:38:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:01.533928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab92eaf-f828-45a5-b78d-4302edadb9de None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:01.539495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58fd78-d2fd-4d61-a443-153712c49707 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:01.643302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab92eaf-f828-45a5-b78d-4302edadb9de None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:01.655033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58fd78-d2fd-4d61-a443-153712c49707 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:01.679915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cea1f-a454-4ace-99ad-38acfcbf0481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:01.680805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cea1f-a454-4ace-99ad-38acfcbf0481 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:01.681088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cea1f-a454-4ace-99ad-38acfcbf0481 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:01.681379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cea1f-a454-4ace-99ad-38acfcbf0481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:01.682226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cea1f-a454-4ace-99ad-38acfcbf0481 None admin] RBAC: Adding query filter params (role.id=7a222a4a0d3b41b08325b7ad1e9b6322, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=2f73ecf1501a4ab181ad9a24ac54aec8) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:01.682473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cea1f-a454-4ace-99ad-38acfcbf0481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:01.683935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cea1f-a454-4ace-99ad-38acfcbf0481 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:01.698762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 499/988] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:01 2022] GET /identity/v3/role_assignments?user.id=2f73ecf1501a4ab181ad9a24ac54aec8&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=7a222a4a0d3b41b08325b7ad1e9b6322 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:01.698978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be509b02-bf19-4f7d-a1db-7f2f92484838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:01.698978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be509b02-bf19-4f7d-a1db-7f2f92484838 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:01.698978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be509b02-bf19-4f7d-a1db-7f2f92484838 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:01.698978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be509b02-bf19-4f7d-a1db-7f2f92484838 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:01.699340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be509b02-bf19-4f7d-a1db-7f2f92484838 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:01.699340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be509b02-bf19-4f7d-a1db-7f2f92484838 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:01.715333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 490/989] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:38:01 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:38:03.437406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a57622-db24-4126-a47f-3eee0643744b None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:03.437924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a57622-db24-4126-a47f-3eee0643744b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:03.438002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a57622-db24-4126-a47f-3eee0643744b None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:03.439653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 500/990] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:03.447053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c47dde-067e-4bf4-bbd7-8a8875630585 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:03.447053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c47dde-067e-4bf4-bbd7-8a8875630585 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:03.447053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c47dde-067e-4bf4-bbd7-8a8875630585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:03.475066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-23c47dde-067e-4bf4-bbd7-8a8875630585 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:03.476544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-23c47dde-067e-4bf4-bbd7-8a8875630585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:03.502323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:03.567088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 491/991] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:03.585281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a57622-db24-4126-a47f-3eee0643744b None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:03.622774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:03.696400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a57622-db24-4126-a47f-3eee0643744b None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:03.732083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfcd0b-a6b9-460e-8c53-0e8242b76b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:03.732886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfcd0b-a6b9-460e-8c53-0e8242b76b1a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:03.733225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfcd0b-a6b9-460e-8c53-0e8242b76b1a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:03.733503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfcd0b-a6b9-460e-8c53-0e8242b76b1a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:03.734204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfcd0b-a6b9-460e-8c53-0e8242b76b1a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:03.735609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfcd0b-a6b9-460e-8c53-0e8242b76b1a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-79cfcd0b-a6b9-460e-8c53-0e8242b76b1a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:03.748470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:03.749485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:03.751444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 501/992] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Jun 15 15:38:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:03.759187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c47dde-067e-4bf4-bbd7-8a8875630585 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:03.859527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32651c6-0a4f-448f-8523-e3543b07749d None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:03.859852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32651c6-0a4f-448f-8523-e3543b07749d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:03.860100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32651c6-0a4f-448f-8523-e3543b07749d None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:03.861437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 502/993] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:03.866125 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f7528-95c0-4dcb-8454-84572243e2bc None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:03.866411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f7528-95c0-4dcb-8454-84572243e2bc None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:03.866685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f7528-95c0-4dcb-8454-84572243e2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:03.868492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c47dde-067e-4bf4-bbd7-8a8875630585 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:03.883667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-759f7528-95c0-4dcb-8454-84572243e2bc None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:03.884838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-759f7528-95c0-4dcb-8454-84572243e2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:03.903799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a7601-ff1c-4bb4-ad27-29f853ac0e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:03.904460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a7601-ff1c-4bb4-ad27-29f853ac0e9a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:03.904746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a7601-ff1c-4bb4-ad27-29f853ac0e9a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:03.905025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a7601-ff1c-4bb4-ad27-29f853ac0e9a None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:03.905733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a7601-ff1c-4bb4-ad27-29f853ac0e9a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:03.906040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a7601-ff1c-4bb4-ad27-29f853ac0e9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:03.907423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a7601-ff1c-4bb4-ad27-29f853ac0e9a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:03.912110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 492/994] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:38:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:03.913359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab6d8-1211-48d7-a9b8-fd76c6855696 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:03.913675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab6d8-1211-48d7-a9b8-fd76c6855696 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:03.913926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab6d8-1211-48d7-a9b8-fd76c6855696 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:03.915177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 493/995] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:03.917444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ab6d8-1211-48d7-a9b8-fd76c6855696 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:03.935808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:03.952872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 503/996] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:38:03.954140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a3789-1d93-405d-81f4-1dbe57c35d9f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:03.954436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a3789-1d93-405d-81f4-1dbe57c35d9f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:03.954711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a3789-1d93-405d-81f4-1dbe57c35d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:03.976840 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-1f6a3789-1d93-405d-81f4-1dbe57c35d9f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:03.978048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1f6a3789-1d93-405d-81f4-1dbe57c35d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:04.002229 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:04.021495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ab6d8-1211-48d7-a9b8-fd76c6855696 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:04.046059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 504/997] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:04.046701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a3789-1d93-405d-81f4-1dbe57c35d9f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:04.055411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ac9f5-85aa-4f5d-8a8a-08e0c3f9b2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:04.056069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ac9f5-85aa-4f5d-8a8a-08e0c3f9b2c6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:04.056353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ac9f5-85aa-4f5d-8a8a-08e0c3f9b2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:04.056628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ac9f5-85aa-4f5d-8a8a-08e0c3f9b2c6 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:04.057327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ac9f5-85aa-4f5d-8a8a-08e0c3f9b2c6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:04.057632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ac9f5-85aa-4f5d-8a8a-08e0c3f9b2c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:04.059011 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ac9f5-85aa-4f5d-8a8a-08e0c3f9b2c6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:04.069760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 494/998] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Jun 15 15:38:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:04.069915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ac9f5-85aa-4f5d-8a8a-08e0c3f9b2c6 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:04.148625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:04.156570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a3789-1d93-405d-81f4-1dbe57c35d9f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:04.168340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ac9f5-85aa-4f5d-8a8a-08e0c3f9b2c6 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:04.189805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74d278-d0c2-4fd8-91bd-58a2e1c72fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:04.190591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74d278-d0c2-4fd8-91bd-58a2e1c72fea None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:04.190911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74d278-d0c2-4fd8-91bd-58a2e1c72fea None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:04.191247 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74d278-d0c2-4fd8-91bd-58a2e1c72fea None admin] PATH_INFO: `/v3/services/image` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:04.191897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74d278-d0c2-4fd8-91bd-58a2e1c72fea None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:04.193369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74d278-d0c2-4fd8-91bd-58a2e1c72fea None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:04.202643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d9c80-9c7f-42c6-a329-aedc52e47ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:04.203572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d9c80-9c7f-42c6-a329-aedc52e47ba3 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:04.203806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d9c80-9c7f-42c6-a329-aedc52e47ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:04.204091 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d9c80-9c7f-42c6-a329-aedc52e47ba3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:04.204742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d9c80-9c7f-42c6-a329-aedc52e47ba3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:04.206087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d9c80-9c7f-42c6-a329-aedc52e47ba3 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-be74d278-d0c2-4fd8-91bd-58a2e1c72fea None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:04.206218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:04.207744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:04.208568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 505/999] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Jun 15 15:38:04 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:04.209358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74d278-d0c2-4fd8-91bd-58a2e1c72fea None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-725d9c80-9c7f-42c6-a329-aedc52e47ba3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:04.221477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:04.223332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:04.224323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 495/1000] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:38:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:04.225155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d9c80-9c7f-42c6-a329-aedc52e47ba3 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:04.238484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:04.317744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74d278-d0c2-4fd8-91bd-58a2e1c72fea None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:04.321432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d9c80-9c7f-42c6-a329-aedc52e47ba3 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:04.351731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0483dc-0ed5-4092-bdb8-8a91c40d79f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:04.352379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0483dc-0ed5-4092-bdb8-8a91c40d79f0 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:04.352658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0483dc-0ed5-4092-bdb8-8a91c40d79f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:04.352931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0483dc-0ed5-4092-bdb8-8a91c40d79f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:04.353657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0483dc-0ed5-4092-bdb8-8a91c40d79f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:04.355047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0483dc-0ed5-4092-bdb8-8a91c40d79f0 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:04.355293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2bfb9-bb68-4b35-9240-cb75d05da44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:04.355951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2bfb9-bb68-4b35-9240-cb75d05da44b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:04.356237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2bfb9-bb68-4b35-9240-cb75d05da44b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:04.356517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2bfb9-bb68-4b35-9240-cb75d05da44b None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:04.357223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2bfb9-bb68-4b35-9240-cb75d05da44b None admin] RBAC: Adding query filter params (name=image) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:04.357536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2bfb9-bb68-4b35-9240-cb75d05da44b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:04.358975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2bfb9-bb68-4b35-9240-cb75d05da44b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:04.364801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 496/1001] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:04.365634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2bfb9-bb68-4b35-9240-cb75d05da44b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:04.386770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 506/1002] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Jun 15 15:38:04 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:04.386770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0483dc-0ed5-4092-bdb8-8a91c40d79f0 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:04.387094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:04.492965 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:04.494302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2bfb9-bb68-4b35-9240-cb75d05da44b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:04.520346 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0483dc-0ed5-4092-bdb8-8a91c40d79f0 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:04.535748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f3f9f-52c8-455b-9357-fcf0f3399d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:04.536431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f3f9f-52c8-455b-9357-fcf0f3399d17 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:04.536717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f3f9f-52c8-455b-9357-fcf0f3399d17 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:04.536995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f3f9f-52c8-455b-9357-fcf0f3399d17 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:04.537703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f3f9f-52c8-455b-9357-fcf0f3399d17 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:04.538010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f3f9f-52c8-455b-9357-fcf0f3399d17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:04.539401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f3f9f-52c8-455b-9357-fcf0f3399d17 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:04.544757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 497/1003] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:38:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:04.550169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f3f9f-52c8-455b-9357-fcf0f3399d17 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:04.557767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241fc6f-25b9-4063-8993-52ddd5476315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:04.559064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241fc6f-25b9-4063-8993-52ddd5476315 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:04.559064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241fc6f-25b9-4063-8993-52ddd5476315 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:04.559064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241fc6f-25b9-4063-8993-52ddd5476315 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:04.559733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241fc6f-25b9-4063-8993-52ddd5476315 None admin] RBAC: Adding query filter params (type=image) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:04.560046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241fc6f-25b9-4063-8993-52ddd5476315 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:04.561964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241fc6f-25b9-4063-8993-52ddd5476315 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:04.566542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 507/1004] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:04.629627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:04.664977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f3f9f-52c8-455b-9357-fcf0f3399d17 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:04.709241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9ca91-567f-422e-b5fc-05f6da3380b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:04.709952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9ca91-567f-422e-b5fc-05f6da3380b0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:04.710238 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9ca91-567f-422e-b5fc-05f6da3380b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:04.710517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9ca91-567f-422e-b5fc-05f6da3380b0 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:04.711297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9ca91-567f-422e-b5fc-05f6da3380b0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:04.711611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9ca91-567f-422e-b5fc-05f6da3380b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:04.714032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9ca91-567f-422e-b5fc-05f6da3380b0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:04.720775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 498/1005] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:38:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:04.727502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241fc6f-25b9-4063-8993-52ddd5476315 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:04.834245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241fc6f-25b9-4063-8993-52ddd5476315 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:04.869249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050222f-4ca1-4909-a604-f705d71b538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:04.869898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050222f-4ca1-4909-a604-f705d71b538d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:04.870180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050222f-4ca1-4909-a604-f705d71b538d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:04.870453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050222f-4ca1-4909-a604-f705d71b538d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:04.871273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050222f-4ca1-4909-a604-f705d71b538d None admin] RBAC: Adding query filter params (interface=public, service_id=414cb4d050534b45a5623fbe851068ff, region_id=RegionOne) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:04.871585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050222f-4ca1-4909-a604-f705d71b538d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:04.872954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050222f-4ca1-4909-a604-f705d71b538d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:04.878347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 508/1006] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Jun 15 15:38:04 2022] GET /identity/v3/endpoints?service_id=414cb4d050534b45a5623fbe851068ff&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:04.884057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9ca91-567f-422e-b5fc-05f6da3380b0 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:04.990140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9ca91-567f-422e-b5fc-05f6da3380b0 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:05.024083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c72eb0-f0e0-4461-83da-bfeb64781183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:05.024743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c72eb0-f0e0-4461-83da-bfeb64781183 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:05.025029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c72eb0-f0e0-4461-83da-bfeb64781183 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:05.025308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c72eb0-f0e0-4461-83da-bfeb64781183 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:05.025984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c72eb0-f0e0-4461-83da-bfeb64781183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:05.027375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c72eb0-f0e0-4461-83da-bfeb64781183 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:05.029137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:05.038819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 499/1007] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Jun 15 15:38:04 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:38:06.543943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06c4bb-a0d8-4ab9-9ad3-119d0f8b27d2 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:06.544449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06c4bb-a0d8-4ab9-9ad3-119d0f8b27d2 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:06.544526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06c4bb-a0d8-4ab9-9ad3-119d0f8b27d2 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:06.545989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 509/1008] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:06.551069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762671e3-af96-4e06-b0dc-70b7ab39009a None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:06.551486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762671e3-af96-4e06-b0dc-70b7ab39009a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:06.551689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762671e3-af96-4e06-b0dc-70b7ab39009a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:06.568389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-762671e3-af96-4e06-b0dc-70b7ab39009a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:06.569548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-762671e3-af96-4e06-b0dc-70b7ab39009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:06.637120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 500/1009] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:06 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:06.662102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06c4bb-a0d8-4ab9-9ad3-119d0f8b27d2 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:06.674176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:06.710078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257cda7-3f51-4466-8a16-d38daa7166f7 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:06.710294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257cda7-3f51-4466-8a16-d38daa7166f7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:06.710574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257cda7-3f51-4466-8a16-d38daa7166f7 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:06.712003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 501/1010] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:38:06.716719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8feffc1-f95d-4c0a-9596-5300036a8b72 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:06.717017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8feffc1-f95d-4c0a-9596-5300036a8b72 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:06.717302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8feffc1-f95d-4c0a-9596-5300036a8b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:06.734149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d8feffc1-f95d-4c0a-9596-5300036a8b72 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:06.735387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d8feffc1-f95d-4c0a-9596-5300036a8b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:06.773468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06c4bb-a0d8-4ab9-9ad3-119d0f8b27d2 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:06.806982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 502/1011] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:06 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:38:06.808596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960162a-1c90-4fed-acbf-bbec1ba6fb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:06.809390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960162a-1c90-4fed-acbf-bbec1ba6fb51 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:06.809670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960162a-1c90-4fed-acbf-bbec1ba6fb51 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:06.809943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960162a-1c90-4fed-acbf-bbec1ba6fb51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:06.811126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960162a-1c90-4fed-acbf-bbec1ba6fb51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:06.812552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960162a-1c90-4fed-acbf-bbec1ba6fb51 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:06.818768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 510/1012] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:06.819674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8feffc1-f95d-4c0a-9596-5300036a8b72 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:06.830437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960162a-1c90-4fed-acbf-bbec1ba6fb51 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:06.883691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:06.917475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:06.932054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8feffc1-f95d-4c0a-9596-5300036a8b72 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:06.944746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960162a-1c90-4fed-acbf-bbec1ba6fb51 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:06.969209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26d775-c2cd-4138-b52a-d0c3685b3644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:06.969797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26d775-c2cd-4138-b52a-d0c3685b3644 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:06.970082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26d775-c2cd-4138-b52a-d0c3685b3644 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:06.970367 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26d775-c2cd-4138-b52a-d0c3685b3644 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:06.971117 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26d775-c2cd-4138-b52a-d0c3685b3644 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:06.973174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26d775-c2cd-4138-b52a-d0c3685b3644 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:06.981895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e20ce1-bc46-42a7-bb17-62a66d18b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:06.982555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e20ce1-bc46-42a7-bb17-62a66d18b003 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:06.982867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e20ce1-bc46-42a7-bb17-62a66d18b003 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:06.983142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e20ce1-bc46-42a7-bb17-62a66d18b003 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:06.984285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e20ce1-bc46-42a7-bb17-62a66d18b003 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:06.986046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e20ce1-bc46-42a7-bb17-62a66d18b003 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:07.001404 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 511/1013] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:38:06 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:38:07.005422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 503/1014] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:38:06 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:07.151943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda189f8-8173-43c0-812e-725fbcb516f6 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:07.152321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda189f8-8173-43c0-812e-725fbcb516f6 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:07.152533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda189f8-8173-43c0-812e-725fbcb516f6 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:07.154001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 512/1015] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:07.160254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc440f5-8e46-4a53-982d-921728f06f60 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:07.160633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc440f5-8e46-4a53-982d-921728f06f60 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:07.160993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc440f5-8e46-4a53-982d-921728f06f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:07.180267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-3dc440f5-8e46-4a53-982d-921728f06f60 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:07.181485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3dc440f5-8e46-4a53-982d-921728f06f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:07.263392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 504/1016] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:07 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:07.287105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda189f8-8173-43c0-812e-725fbcb516f6 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:07.390778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda189f8-8173-43c0-812e-725fbcb516f6 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:07.424375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00105c53-b249-46b1-8942-e0d654f4e8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:07.425697 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00105c53-b249-46b1-8942-e0d654f4e8d3 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:07.426007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00105c53-b249-46b1-8942-e0d654f4e8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:07.426285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00105c53-b249-46b1-8942-e0d654f4e8d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:07.426964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00105c53-b249-46b1-8942-e0d654f4e8d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:07.428333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00105c53-b249-46b1-8942-e0d654f4e8d3 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:07.431469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-00105c53-b249-46b1-8942-e0d654f4e8d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:07.450624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:07.451749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 15:38:07.452577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:07.457215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 513/1017] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:38:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:07.458227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc440f5-8e46-4a53-982d-921728f06f60 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:07.498108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:07.560772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc440f5-8e46-4a53-982d-921728f06f60 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:07.594114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd6d08-05fa-4f8c-830b-be665f1c9e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:07.594814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd6d08-05fa-4f8c-830b-be665f1c9e92 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:07.595155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd6d08-05fa-4f8c-830b-be665f1c9e92 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:07.595437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd6d08-05fa-4f8c-830b-be665f1c9e92 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:07.596139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd6d08-05fa-4f8c-830b-be665f1c9e92 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:07.596447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd6d08-05fa-4f8c-830b-be665f1c9e92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:07.598332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd6d08-05fa-4f8c-830b-be665f1c9e92 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:07.609232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 505/1018] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:38:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:07.609386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00105c53-b249-46b1-8942-e0d654f4e8d3 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:07.710743 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00105c53-b249-46b1-8942-e0d654f4e8d3 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:07.744174 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cdb9b-abf7-4cf5-89ea-58aa5121cab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:07.744843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cdb9b-abf7-4cf5-89ea-58aa5121cab1 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:07.745124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cdb9b-abf7-4cf5-89ea-58aa5121cab1 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:07.745399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cdb9b-abf7-4cf5-89ea-58aa5121cab1 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:07.746102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cdb9b-abf7-4cf5-89ea-58aa5121cab1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:07.746407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cdb9b-abf7-4cf5-89ea-58aa5121cab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:07.747802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cdb9b-abf7-4cf5-89ea-58aa5121cab1 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:07.757517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 514/1019] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Jun 15 15:38:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:07.758931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd6d08-05fa-4f8c-830b-be665f1c9e92 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:07.860007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd6d08-05fa-4f8c-830b-be665f1c9e92 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:07.861463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:07.896157 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef355a-1d7c-4881-8b05-2cf1d0ca6a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:07.896813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef355a-1d7c-4881-8b05-2cf1d0ca6a31 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:07.897099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef355a-1d7c-4881-8b05-2cf1d0ca6a31 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:07.897374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef355a-1d7c-4881-8b05-2cf1d0ca6a31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:07.898102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef355a-1d7c-4881-8b05-2cf1d0ca6a31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:07.899492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef355a-1d7c-4881-8b05-2cf1d0ca6a31 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:07.926760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 506/1020] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Jun 15 15:38:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:09.165812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2b84c-ef63-4bf0-b6b3-07ea328d1b07 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.166350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2b84c-ef63-4bf0-b6b3-07ea328d1b07 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.166350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2b84c-ef63-4bf0-b6b3-07ea328d1b07 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.168002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 515/1021] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:09.173759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a154f-b0f7-4526-9ab2-ed04b7ee6522 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.174268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a154f-b0f7-4526-9ab2-ed04b7ee6522 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.174344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a154f-b0f7-4526-9ab2-ed04b7ee6522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.193081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f96a154f-b0f7-4526-9ab2-ed04b7ee6522 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:09.194360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f96a154f-b0f7-4526-9ab2-ed04b7ee6522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:09.231777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b31a07-4bd9-40b5-97cf-6064c87f469b None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.232089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b31a07-4bd9-40b5-97cf-6064c87f469b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.232326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b31a07-4bd9-40b5-97cf-6064c87f469b None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.233814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 516/1022] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:38:09.239878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3495d0-72e2-42f4-860a-f748ace94498 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.240188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3495d0-72e2-42f4-860a-f748ace94498 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.240466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3495d0-72e2-42f4-860a-f748ace94498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.260069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5a3495d0-72e2-42f4-860a-f748ace94498 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:09.261536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5a3495d0-72e2-42f4-860a-f748ace94498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:09.273381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 507/1023] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:09.295661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a154f-b0f7-4526-9ab2-ed04b7ee6522 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:09.321333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:09.330478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 517/1024] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:09.354472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3495d0-72e2-42f4-860a-f748ace94498 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:09.386934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:09.404982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a154f-b0f7-4526-9ab2-ed04b7ee6522 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:09.443957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084ee2a-d30d-4f8f-a7eb-381e5f5c6fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:09.444673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084ee2a-d30d-4f8f-a7eb-381e5f5c6fca None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.444953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084ee2a-d30d-4f8f-a7eb-381e5f5c6fca None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.445228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084ee2a-d30d-4f8f-a7eb-381e5f5c6fca None admin] PATH_INFO: `/v3/services/image` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.445882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084ee2a-d30d-4f8f-a7eb-381e5f5c6fca None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:09.447284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084ee2a-d30d-4f8f-a7eb-381e5f5c6fca None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-a084ee2a-d30d-4f8f-a7eb-381e5f5c6fca None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:09.458845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:09.460027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:09.460901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 508/1025] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Jun 15 15:38:09 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:09.465789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084ee2a-d30d-4f8f-a7eb-381e5f5c6fca None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:09.472586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3495d0-72e2-42f4-860a-f748ace94498 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:09.508127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7067bb-ed9f-44c4-85a9-a4a90faefa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:09.508924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7067bb-ed9f-44c4-85a9-a4a90faefa77 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.509207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7067bb-ed9f-44c4-85a9-a4a90faefa77 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.509481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7067bb-ed9f-44c4-85a9-a4a90faefa77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.510663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7067bb-ed9f-44c4-85a9-a4a90faefa77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:09.512127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7067bb-ed9f-44c4-85a9-a4a90faefa77 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:09.513732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 518/1026] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:09.519970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7067bb-ed9f-44c4-85a9-a4a90faefa77 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:09.573408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084ee2a-d30d-4f8f-a7eb-381e5f5c6fca None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:09.586492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:09.609702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e799ad-0761-4cca-86c1-44f93187b815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:09.610361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e799ad-0761-4cca-86c1-44f93187b815 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.610643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e799ad-0761-4cca-86c1-44f93187b815 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.610945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e799ad-0761-4cca-86c1-44f93187b815 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.611641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e799ad-0761-4cca-86c1-44f93187b815 None admin] RBAC: Adding query filter params (name=image) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:09.611949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e799ad-0761-4cca-86c1-44f93187b815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:09.613514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e799ad-0761-4cca-86c1-44f93187b815 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:09.618393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 509/1027] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:09.623975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e799ad-0761-4cca-86c1-44f93187b815 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:09.634055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7067bb-ed9f-44c4-85a9-a4a90faefa77 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:09.640139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:09.671066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36208f6d-d357-45ad-ac68-dd271cdc03b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:09.671865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36208f6d-d357-45ad-ac68-dd271cdc03b7 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.672047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36208f6d-d357-45ad-ac68-dd271cdc03b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.672275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36208f6d-d357-45ad-ac68-dd271cdc03b7 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.673000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36208f6d-d357-45ad-ac68-dd271cdc03b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:09.674400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36208f6d-d357-45ad-ac68-dd271cdc03b7 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:09.706976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 519/1028] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:38:09 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:38:09.746507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e799ad-0761-4cca-86c1-44f93187b815 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:09.785842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fae87-9f7c-4940-bf18-3e4078046d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:09.786544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fae87-9f7c-4940-bf18-3e4078046d05 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.786882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fae87-9f7c-4940-bf18-3e4078046d05 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.787170 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fae87-9f7c-4940-bf18-3e4078046d05 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.787897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fae87-9f7c-4940-bf18-3e4078046d05 None admin] RBAC: Adding query filter params (type=image) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:09.788213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fae87-9f7c-4940-bf18-3e4078046d05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:09.789652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fae87-9f7c-4940-bf18-3e4078046d05 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:09.799130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 510/1029] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:09.801566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36208f6d-d357-45ad-ac68-dd271cdc03b7 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:09.909948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36208f6d-d357-45ad-ac68-dd271cdc03b7 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:09.928572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:09.948185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501e565-dd69-4a1e-9f31-93b3437b0bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:09.948848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501e565-dd69-4a1e-9f31-93b3437b0bd9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:09.949134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501e565-dd69-4a1e-9f31-93b3437b0bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:09.949419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501e565-dd69-4a1e-9f31-93b3437b0bd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:09.950211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501e565-dd69-4a1e-9f31-93b3437b0bd9 None admin] RBAC: Adding query filter params (interface=public, service_id=6d86ed3ded0c46a29c8feaecff31fb1d, region_id=RegionOne) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:09.950526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501e565-dd69-4a1e-9f31-93b3437b0bd9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:09.951944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501e565-dd69-4a1e-9f31-93b3437b0bd9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:09.957266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 520/1030] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Jun 15 15:38:09 2022] GET /identity/v3/endpoints?service_id=6d86ed3ded0c46a29c8feaecff31fb1d&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:09.963602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fae87-9f7c-4940-bf18-3e4078046d05 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:10.046144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:10.071137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fae87-9f7c-4940-bf18-3e4078046d05 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:10.105560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222e81d-0a7e-47c0-ab96-69806202b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:10.106237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222e81d-0a7e-47c0-ab96-69806202b326 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:10.106522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222e81d-0a7e-47c0-ab96-69806202b326 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:10.106830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222e81d-0a7e-47c0-ab96-69806202b326 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:10.107524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222e81d-0a7e-47c0-ab96-69806202b326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:10.108923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222e81d-0a7e-47c0-ab96-69806202b326 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:10.116629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 511/1031] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Jun 15 15:38:09 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:38:10.187257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9387aa9-81a2-47d3-ad58-1019915b99a5 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:10.187808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9387aa9-81a2-47d3-ad58-1019915b99a5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:10.187808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9387aa9-81a2-47d3-ad58-1019915b99a5 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:10.189375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 521/1032] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:10.195475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a369cb-da68-4267-a143-13db07c50ee6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:10.196032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a369cb-da68-4267-a143-13db07c50ee6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:10.196032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a369cb-da68-4267-a143-13db07c50ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:10.213901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-34a369cb-da68-4267-a143-13db07c50ee6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:10.215153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-34a369cb-da68-4267-a143-13db07c50ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:10.291229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 512/1033] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:10.315030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9387aa9-81a2-47d3-ad58-1019915b99a5 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:10.418667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9387aa9-81a2-47d3-ad58-1019915b99a5 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:10.462282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054639-4b1c-477c-b845-08cb19486ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:10.463131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054639-4b1c-477c-b845-08cb19486ba4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:10.463419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054639-4b1c-477c-b845-08cb19486ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:10.463695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054639-4b1c-477c-b845-08cb19486ba4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:10.475878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef054639-4b1c-477c-b845-08cb19486ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-ef054639-4b1c-477c-b845-08cb19486ba4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:10.484029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:10.484997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:38:10.485861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:10.487245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 522/1034] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Jun 15 15:38:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:10.491981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a369cb-da68-4267-a143-13db07c50ee6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:10.569207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:10.602630 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a369cb-da68-4267-a143-13db07c50ee6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:10.637406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b00b50-f149-4d9d-af1b-7f6746fe09f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:10.638069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b00b50-f149-4d9d-af1b-7f6746fe09f7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:10.638355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b00b50-f149-4d9d-af1b-7f6746fe09f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:10.638631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b00b50-f149-4d9d-af1b-7f6746fe09f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:10.639410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b00b50-f149-4d9d-af1b-7f6746fe09f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:10.639719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b00b50-f149-4d9d-af1b-7f6746fe09f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:10.641081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b00b50-f149-4d9d-af1b-7f6746fe09f7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:10.650391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 513/1035] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Jun 15 15:38:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:10.655816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054639-4b1c-477c-b845-08cb19486ba4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:10.699996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:10.758685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054639-4b1c-477c-b845-08cb19486ba4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:10.792991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155e3a8-dcb9-4c38-9665-344375fa4dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:10.793782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155e3a8-dcb9-4c38-9665-344375fa4dbc None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:10.794062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155e3a8-dcb9-4c38-9665-344375fa4dbc None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:10.794331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155e3a8-dcb9-4c38-9665-344375fa4dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:10.804849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155e3a8-dcb9-4c38-9665-344375fa4dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:10.806291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155e3a8-dcb9-4c38-9665-344375fa4dbc None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-6155e3a8-dcb9-4c38-9665-344375fa4dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:38:10.823891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:10.824889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:10.830492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:38:10.830492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:38:10.830492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:10.830492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:10.830492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 523/1036] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:10.830808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b00b50-f149-4d9d-af1b-7f6746fe09f7 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:10.944863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b00b50-f149-4d9d-af1b-7f6746fe09f7 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:10.973183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:10.979188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d196b-8a84-49b2-94a9-ba5b52409412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:10.979849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d196b-8a84-49b2-94a9-ba5b52409412 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:10.980133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d196b-8a84-49b2-94a9-ba5b52409412 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:10.980409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d196b-8a84-49b2-94a9-ba5b52409412 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:10.981127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d196b-8a84-49b2-94a9-ba5b52409412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:10.981432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d196b-8a84-49b2-94a9-ba5b52409412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:10.982811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d196b-8a84-49b2-94a9-ba5b52409412 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:10.995015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 514/1037] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:11.000768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155e3a8-dcb9-4c38-9665-344375fa4dbc None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:11.095750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:11.103425 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155e3a8-dcb9-4c38-9665-344375fa4dbc None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:11.137317 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354e3b8-2da7-4e4c-8ce1-1bf528706cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:11.138099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e3b8-2da7-4e4c-8ce1-1bf528706cd7 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:11.138381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e3b8-2da7-4e4c-8ce1-1bf528706cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:11.138657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e3b8-2da7-4e4c-8ce1-1bf528706cd7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:11.146340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354e3b8-2da7-4e4c-8ce1-1bf528706cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:11.147811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354e3b8-2da7-4e4c-8ce1-1bf528706cd7 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-c354e3b8-2da7-4e4c-8ce1-1bf528706cd7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:38:11.161608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:11.162628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:38:11.163677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:11.166799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 524/1038] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Jun 15 15:38:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:11.168450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d196b-8a84-49b2-94a9-ba5b52409412 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:11.272374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d196b-8a84-49b2-94a9-ba5b52409412 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:11.307625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ac9f1-bf5e-4f45-800c-f9dacbe469ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:11.308294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ac9f1-bf5e-4f45-800c-f9dacbe469ce None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:11.308576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ac9f1-bf5e-4f45-800c-f9dacbe469ce None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:11.308850 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ac9f1-bf5e-4f45-800c-f9dacbe469ce None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:11.309679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ac9f1-bf5e-4f45-800c-f9dacbe469ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:11.309988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ac9f1-bf5e-4f45-800c-f9dacbe469ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:11.311445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ac9f1-bf5e-4f45-800c-f9dacbe469ce None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:11.333108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 515/1039] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Jun 15 15:38:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:11.334528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354e3b8-2da7-4e4c-8ce1-1bf528706cd7 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:11.438322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354e3b8-2da7-4e4c-8ce1-1bf528706cd7 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:11.474422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3302521-2782-48bc-a6be-9a1f33184f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:11.475102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3302521-2782-48bc-a6be-9a1f33184f2a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:11.475385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3302521-2782-48bc-a6be-9a1f33184f2a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:11.475670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3302521-2782-48bc-a6be-9a1f33184f2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:11.499155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:11.543757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3302521-2782-48bc-a6be-9a1f33184f2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:11.545207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3302521-2782-48bc-a6be-9a1f33184f2a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:11.646782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 525/1040] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:11 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 15:38:11.649078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ac9f1-bf5e-4f45-800c-f9dacbe469ce None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:11.698889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:11.748688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ac9f1-bf5e-4f45-800c-f9dacbe469ce None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:11.782574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbac7f5-2e3b-435a-b3f3-ba6c99eedb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:11.783396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbac7f5-2e3b-435a-b3f3-ba6c99eedb91 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:11.783680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbac7f5-2e3b-435a-b3f3-ba6c99eedb91 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:11.783955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbac7f5-2e3b-435a-b3f3-ba6c99eedb91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:11.794009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbac7f5-2e3b-435a-b3f3-ba6c99eedb91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:11.795467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbac7f5-2e3b-435a-b3f3-ba6c99eedb91 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-2dbac7f5-2e3b-435a-b3f3-ba6c99eedb91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:38:11.812971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:11.813876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:11.814816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:38:11.814816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:38:11.814816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:11.814816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:11.814816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 516/1041] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:11.821231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3302521-2782-48bc-a6be-9a1f33184f2a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:11.881652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:11.914304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3136f-44bb-40d1-8d75-de4139bec8eb None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:11.914615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3136f-44bb-40d1-8d75-de4139bec8eb None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:11.914914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3136f-44bb-40d1-8d75-de4139bec8eb None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:11.916432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 517/1042] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:38:11.921241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a8339-2a2b-479d-a1ff-a9d656209580 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:11.921531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a8339-2a2b-479d-a1ff-a9d656209580 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:11.921812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a8339-2a2b-479d-a1ff-a9d656209580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:11.928377 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3302521-2782-48bc-a6be-9a1f33184f2a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:11.938823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-389a8339-2a2b-479d-a1ff-a9d656209580 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:11.940245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-389a8339-2a2b-479d-a1ff-a9d656209580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:11.962605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139e875-e1d7-48ba-a98b-c5ba32d9789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:11.963227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139e875-e1d7-48ba-a98b-c5ba32d9789d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:11.963508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139e875-e1d7-48ba-a98b-c5ba32d9789d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:11.963786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139e875-e1d7-48ba-a98b-c5ba32d9789d None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:11.964499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139e875-e1d7-48ba-a98b-c5ba32d9789d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:11.964805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139e875-e1d7-48ba-a98b-c5ba32d9789d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:11.966168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139e875-e1d7-48ba-a98b-c5ba32d9789d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:11.977820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 526/1043] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:11.983196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139e875-e1d7-48ba-a98b-c5ba32d9789d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.018775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 518/1044] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:38:12.040970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a8339-2a2b-479d-a1ff-a9d656209580 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.072484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:12.093088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139e875-e1d7-48ba-a98b-c5ba32d9789d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.108835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:12.132461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a280ab-8554-4ba4-b105-70a4dc1dcbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:12.133248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a280ab-8554-4ba4-b105-70a4dc1dcbde None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.133525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a280ab-8554-4ba4-b105-70a4dc1dcbde None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.133819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a280ab-8554-4ba4-b105-70a4dc1dcbde None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.140442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a280ab-8554-4ba4-b105-70a4dc1dcbde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:12.141879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a280ab-8554-4ba4-b105-70a4dc1dcbde None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:12.144778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a8339-2a2b-479d-a1ff-a9d656209580 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-97a280ab-8554-4ba4-b105-70a4dc1dcbde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:38:12.156480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:12.157391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:12.158461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:38:12.158461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:38:12.158461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:38:12.158461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:38:12.158461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:38:12.158461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:38:12.158461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:38:12.158461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:12.158461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 527/1045] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:38:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:12.163699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a280ab-8554-4ba4-b105-70a4dc1dcbde None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.184247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769be3b3-09fa-444a-ba1a-6ffe3c47a642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:12.185042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769be3b3-09fa-444a-ba1a-6ffe3c47a642 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.185324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769be3b3-09fa-444a-ba1a-6ffe3c47a642 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.185603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769be3b3-09fa-444a-ba1a-6ffe3c47a642 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.196435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769be3b3-09fa-444a-ba1a-6ffe3c47a642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-769be3b3-09fa-444a-ba1a-6ffe3c47a642 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:12.204256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:12.205301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:12.206186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 519/1046] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:12.211985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769be3b3-09fa-444a-ba1a-6ffe3c47a642 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.279036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a280ab-8554-4ba4-b105-70a4dc1dcbde None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.312349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:12.313539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753f148-5e25-436f-b055-7de634b0787e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:12.314158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f148-5e25-436f-b055-7de634b0787e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.314473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f148-5e25-436f-b055-7de634b0787e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.314837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f148-5e25-436f-b055-7de634b0787e None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.316073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753f148-5e25-436f-b055-7de634b0787e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:12.316392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753f148-5e25-436f-b055-7de634b0787e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:12.317828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753f148-5e25-436f-b055-7de634b0787e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:12.322806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769be3b3-09fa-444a-ba1a-6ffe3c47a642 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.329241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 528/1047] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:12.330482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af01d28-2ca8-4137-8bc4-c33db8fb2c93 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.330803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af01d28-2ca8-4137-8bc4-c33db8fb2c93 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.331117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af01d28-2ca8-4137-8bc4-c33db8fb2c93 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.332514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 529/1048] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:12.335029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af01d28-2ca8-4137-8bc4-c33db8fb2c93 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.357746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4a080-186c-44aa-93ef-5057d034e240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:12.358408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a080-186c-44aa-93ef-5057d034e240 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.358691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a080-186c-44aa-93ef-5057d034e240 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.359000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a080-186c-44aa-93ef-5057d034e240 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.359778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4a080-186c-44aa-93ef-5057d034e240 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:12.360090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4a080-186c-44aa-93ef-5057d034e240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:12.361441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4a080-186c-44aa-93ef-5057d034e240 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:12.370009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:12.370505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 520/1049] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:12.371837 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8bb81-13b8-411c-a4e9-8bd42c7a19e0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.372131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8bb81-13b8-411c-a4e9-8bd42c7a19e0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.372408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8bb81-13b8-411c-a4e9-8bd42c7a19e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.389775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0af8bb81-13b8-411c-a4e9-8bd42c7a19e0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:12.390999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0af8bb81-13b8-411c-a4e9-8bd42c7a19e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:12.439925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af01d28-2ca8-4137-8bc4-c33db8fb2c93 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.460887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 521/1050] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:12.461553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8bb81-13b8-411c-a4e9-8bd42c7a19e0 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.479095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee970d03-c052-4f3d-9315-c513eefc1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:12.479765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee970d03-c052-4f3d-9315-c513eefc1de9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.480040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee970d03-c052-4f3d-9315-c513eefc1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.480318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee970d03-c052-4f3d-9315-c513eefc1de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.481141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee970d03-c052-4f3d-9315-c513eefc1de9 None admin] RBAC: Adding query filter params (role.id=7a222a4a0d3b41b08325b7ad1e9b6322, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=4327e4eed5a1436ea1a00c2209bc5750) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:12.481453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee970d03-c052-4f3d-9315-c513eefc1de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:12.483007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee970d03-c052-4f3d-9315-c513eefc1de9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:12.489461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 530/1051] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/role_assignments?user.id=4327e4eed5a1436ea1a00c2209bc5750&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=7a222a4a0d3b41b08325b7ad1e9b6322 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:12.494066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee970d03-c052-4f3d-9315-c513eefc1de9 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.559444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:12.578545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8bb81-13b8-411c-a4e9-8bd42c7a19e0 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.618789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc825894-02c1-4b7f-9857-eb6e44adc81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:12.619062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:12.619789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc825894-02c1-4b7f-9857-eb6e44adc81f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.620075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc825894-02c1-4b7f-9857-eb6e44adc81f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.620348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc825894-02c1-4b7f-9857-eb6e44adc81f None admin] PATH_INFO: `/v3/users/efa89e576bc74765b3b490cd3db23bae` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.623597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee970d03-c052-4f3d-9315-c513eefc1de9 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.636987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc825894-02c1-4b7f-9857-eb6e44adc81f None admin] RBAC: Authorizing `identity:get_user(user_id=efa89e576bc74765b3b490cd3db23bae)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:12.638439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc825894-02c1-4b7f-9857-eb6e44adc81f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:12.640412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 522/1052] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/users/efa89e576bc74765b3b490cd3db23bae => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:12.646363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc825894-02c1-4b7f-9857-eb6e44adc81f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.678556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f0a5d-1e41-4977-ad35-d224cf942a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:12.679667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f0a5d-1e41-4977-ad35-d224cf942a03 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.679951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f0a5d-1e41-4977-ad35-d224cf942a03 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.680226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f0a5d-1e41-4977-ad35-d224cf942a03 None admin] PATH_INFO: `/v3/services/image` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.680885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f0a5d-1e41-4977-ad35-d224cf942a03 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:12.682265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f0a5d-1e41-4977-ad35-d224cf942a03 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-1a8f0a5d-1e41-4977-ad35-d224cf942a03 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:12.694821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:12.695857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:12.696745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 531/1053] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/services/image => generated 86 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:12.702080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f0a5d-1e41-4977-ad35-d224cf942a03 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.752890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc825894-02c1-4b7f-9857-eb6e44adc81f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.789622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f9a65-d8bf-4725-910f-0fc90fe0164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:12.790395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f9a65-d8bf-4725-910f-0fc90fe0164f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.790677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f9a65-d8bf-4725-910f-0fc90fe0164f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.790993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f9a65-d8bf-4725-910f-0fc90fe0164f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.803577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:12.815995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f0a5d-1e41-4977-ad35-d224cf942a03 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.852495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7118f7-3fdf-4879-ab3b-2b029a31f783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:12.853262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7118f7-3fdf-4879-ab3b-2b029a31f783 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:12.853664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7118f7-3fdf-4879-ab3b-2b029a31f783 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:12.853948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7118f7-3fdf-4879-ab3b-2b029a31f783 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:12.854664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7118f7-3fdf-4879-ab3b-2b029a31f783 None admin] RBAC: Adding query filter params (name=image) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:12.855001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7118f7-3fdf-4879-ab3b-2b029a31f783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:12.856375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7118f7-3fdf-4879-ab3b-2b029a31f783 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:12.859948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f9a65-d8bf-4725-910f-0fc90fe0164f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:12.861387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f9a65-d8bf-4725-910f-0fc90fe0164f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:12.866638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 532/1054] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:12.868097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7118f7-3fdf-4879-ab3b-2b029a31f783 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:12.897173 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:12.974893 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 523/1055] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:38:12.976330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7118f7-3fdf-4879-ab3b-2b029a31f783 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:12.980457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f9a65-d8bf-4725-910f-0fc90fe0164f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:13.013943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a938d-9cf7-4de8-91cc-7f9bd40de514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:13.014782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a938d-9cf7-4de8-91cc-7f9bd40de514 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:13.014919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a938d-9cf7-4de8-91cc-7f9bd40de514 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:13.015188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a938d-9cf7-4de8-91cc-7f9bd40de514 None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:13.015888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a938d-9cf7-4de8-91cc-7f9bd40de514 None admin] RBAC: Adding query filter params (type=image) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:13.016201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a938d-9cf7-4de8-91cc-7f9bd40de514 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:13.017579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a938d-9cf7-4de8-91cc-7f9bd40de514 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:13.027749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 533/1056] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:13.028316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a938d-9cf7-4de8-91cc-7f9bd40de514 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:13.031308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:13.092126 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f9a65-d8bf-4725-910f-0fc90fe0164f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:13.119744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:13.129566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523756fe-0528-4ab8-ac58-e4046fa68910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:13.130433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523756fe-0528-4ab8-ac58-e4046fa68910 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:13.130605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a938d-9cf7-4de8-91cc-7f9bd40de514 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:13.130706 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523756fe-0528-4ab8-ac58-e4046fa68910 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:13.131009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523756fe-0528-4ab8-ac58-e4046fa68910 None admin] PATH_INFO: `/v3/projects/d59dea7b3bef496f86bbe3d53591977f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:13.132193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523756fe-0528-4ab8-ac58-e4046fa68910 None admin] RBAC: Authorizing `identity:get_project(project_id=d59dea7b3bef496f86bbe3d53591977f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:13.133627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523756fe-0528-4ab8-ac58-e4046fa68910 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:13.140312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 524/1057] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:12 2022] GET /identity/v3/projects/d59dea7b3bef496f86bbe3d53591977f => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:13.141105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523756fe-0528-4ab8-ac58-e4046fa68910 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:13.172105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018ceb7-cc60-401c-9b25-9bdd1ee56526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:13.172763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018ceb7-cc60-401c-9b25-9bdd1ee56526 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:13.173048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018ceb7-cc60-401c-9b25-9bdd1ee56526 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:13.173326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018ceb7-cc60-401c-9b25-9bdd1ee56526 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:13.174063 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018ceb7-cc60-401c-9b25-9bdd1ee56526 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:13.175555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018ceb7-cc60-401c-9b25-9bdd1ee56526 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:13.199904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 534/1058] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Jun 15 15:38:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:13.266292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523756fe-0528-4ab8-ac58-e4046fa68910 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:13.278229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:13.312371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95391134-6082-4df4-bd6b-62638edf62be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:13.313030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391134-6082-4df4-bd6b-62638edf62be None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:13.313313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391134-6082-4df4-bd6b-62638edf62be None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:13.313591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391134-6082-4df4-bd6b-62638edf62be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:13.314429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95391134-6082-4df4-bd6b-62638edf62be None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=d59dea7b3bef496f86bbe3d53591977f, user.id=efa89e576bc74765b3b490cd3db23bae) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:13.314769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95391134-6082-4df4-bd6b-62638edf62be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:13.316786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95391134-6082-4df4-bd6b-62638edf62be None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:13.331045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 525/1059] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:13 2022] GET /identity/v3/role_assignments?user.id=efa89e576bc74765b3b490cd3db23bae&scope.project.id=d59dea7b3bef496f86bbe3d53591977f&role.id=433478cc318a467c931f13a757f161fa => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:14.628737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee58677-c31c-4de2-9dec-052480de23a6 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:14.629249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee58677-c31c-4de2-9dec-052480de23a6 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:14.629329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee58677-c31c-4de2-9dec-052480de23a6 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:14.630813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 535/1060] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:14.636185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af28ac-a3c7-4c82-9228-ff20b63e01c2 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:14.636578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af28ac-a3c7-4c82-9228-ff20b63e01c2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:14.636771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af28ac-a3c7-4c82-9228-ff20b63e01c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:14.653508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b8af28ac-a3c7-4c82-9228-ff20b63e01c2 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:14.654682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b8af28ac-a3c7-4c82-9228-ff20b63e01c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:14.722511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 526/1061] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:14.746088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee58677-c31c-4de2-9dec-052480de23a6 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:14.850815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee58677-c31c-4de2-9dec-052480de23a6 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:14.891636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db260a-bc51-4b93-ad28-3807584ca80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:14.892530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db260a-bc51-4b93-ad28-3807584ca80a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:14.892794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db260a-bc51-4b93-ad28-3807584ca80a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:14.893070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db260a-bc51-4b93-ad28-3807584ca80a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:14.908544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db260a-bc51-4b93-ad28-3807584ca80a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-32db260a-bc51-4b93-ad28-3807584ca80a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:14.917369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:14.918307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:38:14.919510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:14.923837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 536/1062] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Jun 15 15:38:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:14.925700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af28ac-a3c7-4c82-9228-ff20b63e01c2 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:15.022484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:15.026202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af28ac-a3c7-4c82-9228-ff20b63e01c2 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:15.061304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea67a9e-a6fb-444c-976d-c97c165ce760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:15.061975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea67a9e-a6fb-444c-976d-c97c165ce760 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.062256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea67a9e-a6fb-444c-976d-c97c165ce760 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.062532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea67a9e-a6fb-444c-976d-c97c165ce760 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.063305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea67a9e-a6fb-444c-976d-c97c165ce760 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:15.063614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea67a9e-a6fb-444c-976d-c97c165ce760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:15.065024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea67a9e-a6fb-444c-976d-c97c165ce760 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:15.078149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 527/1063] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Jun 15 15:38:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:15.079595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db260a-bc51-4b93-ad28-3807584ca80a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:15.088687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:15.183200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db260a-bc51-4b93-ad28-3807584ca80a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:15.217712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ef0f8-f572-4a02-b914-d7ae05c553d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:15.218502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ef0f8-f572-4a02-b914-d7ae05c553d3 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.218808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ef0f8-f572-4a02-b914-d7ae05c553d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.219086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ef0f8-f572-4a02-b914-d7ae05c553d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.229869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ef0f8-f572-4a02-b914-d7ae05c553d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:15.231398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ef0f8-f572-4a02-b914-d7ae05c553d3 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-226ef0f8-f572-4a02-b914-d7ae05c553d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:38:15.253920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:15.254935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:15.255944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:38:15.255944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:38:15.255944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:15.255944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:15.258815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 537/1064] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:15.261223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea67a9e-a6fb-444c-976d-c97c165ce760 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:15.365459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499ff25-7c8a-42d7-a11f-7d196d1e762b None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.365658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499ff25-7c8a-42d7-a11f-7d196d1e762b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.365923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499ff25-7c8a-42d7-a11f-7d196d1e762b None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.370779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 538/1065] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:38:15.373259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b108f-a63f-4308-942a-d5558445e8d3 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.373551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b108f-a63f-4308-942a-d5558445e8d3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.373818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b108f-a63f-4308-942a-d5558445e8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.374025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea67a9e-a6fb-444c-976d-c97c165ce760 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:15.391819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4c9b108f-a63f-4308-942a-d5558445e8d3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:15.393059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4c9b108f-a63f-4308-942a-d5558445e8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:15.413734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604ca01-1392-48a3-aaf7-9f7d88dc8fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:15.414416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604ca01-1392-48a3-aaf7-9f7d88dc8fed None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.414754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604ca01-1392-48a3-aaf7-9f7d88dc8fed None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.415033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604ca01-1392-48a3-aaf7-9f7d88dc8fed None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.415758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604ca01-1392-48a3-aaf7-9f7d88dc8fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:15.416076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604ca01-1392-48a3-aaf7-9f7d88dc8fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:15.417441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604ca01-1392-48a3-aaf7-9f7d88dc8fed None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:15.434800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 528/1066] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:15.437004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604ca01-1392-48a3-aaf7-9f7d88dc8fed None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:15.460991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:15.463768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:15.475776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 539/1067] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:15.477064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96af8-c40d-408e-8a0b-6108be72fb53 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.477343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96af8-c40d-408e-8a0b-6108be72fb53 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.477615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96af8-c40d-408e-8a0b-6108be72fb53 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.479019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 540/1068] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:15.484005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa856d-465b-4a08-a5d1-76a51a83ce27 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.484286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa856d-465b-4a08-a5d1-76a51a83ce27 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.484560 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa856d-465b-4a08-a5d1-76a51a83ce27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.500426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-71fa856d-465b-4a08-a5d1-76a51a83ce27 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:15.501594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-71fa856d-465b-4a08-a5d1-76a51a83ce27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:15.543246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604ca01-1392-48a3-aaf7-9f7d88dc8fed None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:15.566250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 541/1069] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:38:15.566927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa856d-465b-4a08-a5d1-76a51a83ce27 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:15.576739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddee46-6f3d-4cb8-8aa1-073c23cf9e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:15.577527 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddee46-6f3d-4cb8-8aa1-073c23cf9e78 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.577804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddee46-6f3d-4cb8-8aa1-073c23cf9e78 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.578073 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddee46-6f3d-4cb8-8aa1-073c23cf9e78 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.586075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddee46-6f3d-4cb8-8aa1-073c23cf9e78 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:15.587556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddee46-6f3d-4cb8-8aa1-073c23cf9e78 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-54ddee46-6f3d-4cb8-8aa1-073c23cf9e78 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:38:15.603066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:15.604108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 529/1070] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Jun 15 15:38:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:15.605064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddee46-6f3d-4cb8-8aa1-073c23cf9e78 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:15.685134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa856d-465b-4a08-a5d1-76a51a83ce27 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:15.686990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:15.703028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:15.705731 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddee46-6f3d-4cb8-8aa1-073c23cf9e78 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:15.720626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd48cf-773e-4e6c-a9e0-e9a9c82739eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:15.721273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd48cf-773e-4e6c-a9e0-e9a9c82739eb None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.721580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd48cf-773e-4e6c-a9e0-e9a9c82739eb None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.721855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd48cf-773e-4e6c-a9e0-e9a9c82739eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.738812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3ab33-efe6-401f-bc7f-6f82f8afcd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:15.739588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3ab33-efe6-401f-bc7f-6f82f8afcd9d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.739867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3ab33-efe6-401f-bc7f-6f82f8afcd9d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.740139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3ab33-efe6-401f-bc7f-6f82f8afcd9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.749875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3ab33-efe6-401f-bc7f-6f82f8afcd9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-0db3ab33-efe6-401f-bc7f-6f82f8afcd9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:15.757912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:15.759010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 530/1071] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:38:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:15.759871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3ab33-efe6-401f-bc7f-6f82f8afcd9d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:15.792610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd48cf-773e-4e6c-a9e0-e9a9c82739eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:15.794057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd48cf-773e-4e6c-a9e0-e9a9c82739eb None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:15.861632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3ab33-efe6-401f-bc7f-6f82f8afcd9d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:15.890792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 542/1072] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:38:15.890792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd48cf-773e-4e6c-a9e0-e9a9c82739eb None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:15.897179 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd0de7-8a14-4998-a704-53529869e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:15.897841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd0de7-8a14-4998-a704-53529869e307 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:15.898120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd0de7-8a14-4998-a704-53529869e307 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:15.898394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd0de7-8a14-4998-a704-53529869e307 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:15.899263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd0de7-8a14-4998-a704-53529869e307 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:15.899575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd0de7-8a14-4998-a704-53529869e307 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:15.900984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd0de7-8a14-4998-a704-53529869e307 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:15.910128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:15.915767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:15.923089 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 531/1073] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Jun 15 15:38:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:15.924018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd0de7-8a14-4998-a704-53529869e307 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:15.990204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd48cf-773e-4e6c-a9e0-e9a9c82739eb None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.019780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd0de7-8a14-4998-a704-53529869e307 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.023144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7f16e-f981-4ac0-a0b5-95e7c407a4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.023804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7f16e-f981-4ac0-a0b5-95e7c407a4a2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.024087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7f16e-f981-4ac0-a0b5-95e7c407a4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.024365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7f16e-f981-4ac0-a0b5-95e7c407a4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.025097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7f16e-f981-4ac0-a0b5-95e7c407a4a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:16.025403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7f16e-f981-4ac0-a0b5-95e7c407a4a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.026798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7f16e-f981-4ac0-a0b5-95e7c407a4a2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.035152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 543/1074] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:38:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:16.035923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7f16e-f981-4ac0-a0b5-95e7c407a4a2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:16.053626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe503e-9100-406b-83e5-f67929cb6f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.054413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe503e-9100-406b-83e5-f67929cb6f82 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.054693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe503e-9100-406b-83e5-f67929cb6f82 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.054994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe503e-9100-406b-83e5-f67929cb6f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.064963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe503e-9100-406b-83e5-f67929cb6f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.066459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe503e-9100-406b-83e5-f67929cb6f82 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-75fe503e-9100-406b-83e5-f67929cb6f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:38:16.084027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:16.085050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:16.085955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:38:16.085955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:38:16.085955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:16.085955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:16.085955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 532/1075] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:16.085955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe503e-9100-406b-83e5-f67929cb6f82 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:16.118275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:16.136919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7f16e-f981-4ac0-a0b5-95e7c407a4a2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.140701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:16.171035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8aaf2-f7ee-4486-ba25-fdc67c0b32d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.171824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8aaf2-f7ee-4486-ba25-fdc67c0b32d3 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.172105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8aaf2-f7ee-4486-ba25-fdc67c0b32d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.172388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8aaf2-f7ee-4486-ba25-fdc67c0b32d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.183963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8aaf2-f7ee-4486-ba25-fdc67c0b32d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.185398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8aaf2-f7ee-4486-ba25-fdc67c0b32d3 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.187814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe503e-9100-406b-83e5-f67929cb6f82 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-f3d8aaf2-f7ee-4486-ba25-fdc67c0b32d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:38:16.202503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:16.203465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:16.204375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:38:16.204375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:38:16.204375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:16.204375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:16.204375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 544/1076] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:16.204694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8aaf2-f7ee-4486-ba25-fdc67c0b32d3 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:16.227124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17625da8-5dd7-4200-b54f-95b85b3b3170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.227994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17625da8-5dd7-4200-b54f-95b85b3b3170 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.228281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17625da8-5dd7-4200-b54f-95b85b3b3170 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.228560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17625da8-5dd7-4200-b54f-95b85b3b3170 None admin] PATH_INFO: `/v3/users/efa89e576bc74765b3b490cd3db23bae` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.229750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17625da8-5dd7-4200-b54f-95b85b3b3170 None admin] RBAC: Authorizing `identity:get_user(user_id=efa89e576bc74765b3b490cd3db23bae)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.231215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17625da8-5dd7-4200-b54f-95b85b3b3170 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.237454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 533/1077] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:16 2022] GET /identity/v3/users/efa89e576bc74765b3b490cd3db23bae => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:16.237454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17625da8-5dd7-4200-b54f-95b85b3b3170 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:16.308392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8aaf2-f7ee-4486-ba25-fdc67c0b32d3 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.332530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17625da8-5dd7-4200-b54f-95b85b3b3170 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.337882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:16.343823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef79328-5e06-46e1-90c2-547f0c304ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.344481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef79328-5e06-46e1-90c2-547f0c304ecc None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.344767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef79328-5e06-46e1-90c2-547f0c304ecc None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.345047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef79328-5e06-46e1-90c2-547f0c304ecc None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.345761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef79328-5e06-46e1-90c2-547f0c304ecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:16.346069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef79328-5e06-46e1-90c2-547f0c304ecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.348027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef79328-5e06-46e1-90c2-547f0c304ecc None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.352797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:16.360118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 545/1078] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:16.360748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef79328-5e06-46e1-90c2-547f0c304ecc None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:16.367886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc27c7-bb26-437c-b06f-14a043e62f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.368571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc27c7-bb26-437c-b06f-14a043e62f97 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.368856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc27c7-bb26-437c-b06f-14a043e62f97 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.369137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc27c7-bb26-437c-b06f-14a043e62f97 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.369880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc27c7-bb26-437c-b06f-14a043e62f97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:16.370191 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc27c7-bb26-437c-b06f-14a043e62f97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.371628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc27c7-bb26-437c-b06f-14a043e62f97 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.384335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 534/1079] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:16.390084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc27c7-bb26-437c-b06f-14a043e62f97 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:16.497897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef79328-5e06-46e1-90c2-547f0c304ecc None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.512716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc27c7-bb26-437c-b06f-14a043e62f97 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.538378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9de10-99f0-4032-b650-244ed283597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.539389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9de10-99f0-4032-b650-244ed283597b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.539624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9de10-99f0-4032-b650-244ed283597b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.539906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9de10-99f0-4032-b650-244ed283597b None admin] PATH_INFO: `/v3/projects/d59dea7b3bef496f86bbe3d53591977f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.541182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9de10-99f0-4032-b650-244ed283597b None admin] RBAC: Authorizing `identity:get_project(project_id=d59dea7b3bef496f86bbe3d53591977f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.542641 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9de10-99f0-4032-b650-244ed283597b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.545103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 546/1080] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:16 2022] GET /identity/v3/projects/d59dea7b3bef496f86bbe3d53591977f => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:16.551831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9de10-99f0-4032-b650-244ed283597b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:16.553612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feafa9-9066-47e0-9d82-cc2d70da5f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.554874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feafa9-9066-47e0-9d82-cc2d70da5f69 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.554874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feafa9-9066-47e0-9d82-cc2d70da5f69 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.555045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feafa9-9066-47e0-9d82-cc2d70da5f69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.562957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1feafa9-9066-47e0-9d82-cc2d70da5f69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.564707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1feafa9-9066-47e0-9d82-cc2d70da5f69 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-d1feafa9-9066-47e0-9d82-cc2d70da5f69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:38:16.579300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:16.580253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:16.581241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:38:16.581241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:38:16.581241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:38:16.581241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:38:16.581241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:38:16.581241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:38:16.581241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:38:16.581241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:16.582209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 535/1081] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:16.586404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feafa9-9066-47e0-9d82-cc2d70da5f69 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:16.618655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:16.637517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:16.668329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9de10-99f0-4032-b650-244ed283597b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.700512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feafa9-9066-47e0-9d82-cc2d70da5f69 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.702615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8674569-1398-405a-8757-8f5cd45ee64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.703548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8674569-1398-405a-8757-8f5cd45ee64f None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.703830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8674569-1398-405a-8757-8f5cd45ee64f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.704107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8674569-1398-405a-8757-8f5cd45ee64f None admin] PATH_INFO: `/v3/projects/d59dea7b3bef496f86bbe3d53591977f/users/efa89e576bc74765b3b490cd3db23bae/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.706641 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8674569-1398-405a-8757-8f5cd45ee64f None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dea7b3bef496f86bbe3d53591977f, user_id=efa89e576bc74765b3b490cd3db23bae, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.708112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8674569-1398-405a-8757-8f5cd45ee64f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.720622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 547/1082] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:38:16 2022] PUT /identity/v3/projects/d59dea7b3bef496f86bbe3d53591977f/users/efa89e576bc74765b3b490cd3db23bae/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:38:16.735352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2749cbf-7a66-4b9b-b9dc-116270f9e422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.736041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2749cbf-7a66-4b9b-b9dc-116270f9e422 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.736322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2749cbf-7a66-4b9b-b9dc-116270f9e422 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.736593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2749cbf-7a66-4b9b-b9dc-116270f9e422 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.737432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2749cbf-7a66-4b9b-b9dc-116270f9e422 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:16.737738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2749cbf-7a66-4b9b-b9dc-116270f9e422 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.739231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2749cbf-7a66-4b9b-b9dc-116270f9e422 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.757928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 536/1083] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:16.759033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8674569-1398-405a-8757-8f5cd45ee64f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:16.864544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8674569-1398-405a-8757-8f5cd45ee64f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:16.886357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:16.902941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be930f5-7b7f-4cbc-a8f2-c5e8da5a6a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:16.903844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be930f5-7b7f-4cbc-a8f2-c5e8da5a6a81 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:16.904126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be930f5-7b7f-4cbc-a8f2-c5e8da5a6a81 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:16.904402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be930f5-7b7f-4cbc-a8f2-c5e8da5a6a81 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/4327e4eed5a1436ea1a00c2209bc5750/roles/7a222a4a0d3b41b08325b7ad1e9b6322` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:16.906625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be930f5-7b7f-4cbc-a8f2-c5e8da5a6a81 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=4327e4eed5a1436ea1a00c2209bc5750, role_id=7a222a4a0d3b41b08325b7ad1e9b6322)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:16.908092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be930f5-7b7f-4cbc-a8f2-c5e8da5a6a81 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:16.921680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 548/1084] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:38:16 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/4327e4eed5a1436ea1a00c2209bc5750/roles/7a222a4a0d3b41b08325b7ad1e9b6322 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:38:18.821687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b73f1b-a138-4bac-9920-52ea3754e590 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:18.822203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b73f1b-a138-4bac-9920-52ea3754e590 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:18.822203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b73f1b-a138-4bac-9920-52ea3754e590 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:18.823562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 537/1085] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:18.828647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9efdde-abea-423a-bde0-f3a3056341d4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:18.829060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9efdde-abea-423a-bde0-f3a3056341d4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:18.829195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9efdde-abea-423a-bde0-f3a3056341d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:18.845973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2c9efdde-abea-423a-bde0-f3a3056341d4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:18.847213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2c9efdde-abea-423a-bde0-f3a3056341d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:18.915262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 549/1086] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:18.943668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b73f1b-a138-4bac-9920-52ea3754e590 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.013986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1723-0424-4986-bb59-12a5160e4e0a None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.014266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1723-0424-4986-bb59-12a5160e4e0a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.014545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1723-0424-4986-bb59-12a5160e4e0a None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.016072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 550/1087] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:19.021165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a23ad-dc9b-4847-9743-1c1e2d3d220a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.021455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a23ad-dc9b-4847-9743-1c1e2d3d220a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.021736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a23ad-dc9b-4847-9743-1c1e2d3d220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.039096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9a0a23ad-dc9b-4847-9743-1c1e2d3d220a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:19.040273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9a0a23ad-dc9b-4847-9743-1c1e2d3d220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:19.043755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:19.048037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b73f1b-a138-4bac-9920-52ea3754e590 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.082502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d9cd-fdbb-4c07-9e64-79115d81ea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.083251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d9cd-fdbb-4c07-9e64-79115d81ea27 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.083543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d9cd-fdbb-4c07-9e64-79115d81ea27 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.083816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d9cd-fdbb-4c07-9e64-79115d81ea27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.093604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350d9cd-fdbb-4c07-9e64-79115d81ea27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-a350d9cd-fdbb-4c07-9e64-79115d81ea27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:19.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:19.102958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:19.104027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 538/1088] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:38:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:19.108768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d9cd-fdbb-4c07-9e64-79115d81ea27 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.109225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 551/1089] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:38:19.132203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a23ad-dc9b-4847-9743-1c1e2d3d220a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.174399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:19.207753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d9cd-fdbb-4c07-9e64-79115d81ea27 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.232799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a23ad-dc9b-4847-9743-1c1e2d3d220a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.241299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784fe0-e37c-4683-a5f4-dadbe2fb100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.241958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784fe0-e37c-4683-a5f4-dadbe2fb100c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.242235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784fe0-e37c-4683-a5f4-dadbe2fb100c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.242506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784fe0-e37c-4683-a5f4-dadbe2fb100c None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.243282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10784fe0-e37c-4683-a5f4-dadbe2fb100c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:19.243584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10784fe0-e37c-4683-a5f4-dadbe2fb100c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:19.244932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10784fe0-e37c-4683-a5f4-dadbe2fb100c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:19.252819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 539/1090] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:19.258595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784fe0-e37c-4683-a5f4-dadbe2fb100c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.265508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb725d-d85d-4f27-9698-13ce910ddd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.266311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb725d-d85d-4f27-9698-13ce910ddd02 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.266595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb725d-d85d-4f27-9698-13ce910ddd02 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.266919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb725d-d85d-4f27-9698-13ce910ddd02 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.277738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb725d-d85d-4f27-9698-13ce910ddd02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:19.282918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-01bb725d-d85d-4f27-9698-13ce910ddd02 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:19.286445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:19.287493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:19.288575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 552/1091] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:19.293410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb725d-d85d-4f27-9698-13ce910ddd02 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.360426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784fe0-e37c-4683-a5f4-dadbe2fb100c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.387391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:19.390367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb725d-d85d-4f27-9698-13ce910ddd02 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.394638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501400c-b957-48e4-b5bd-813cac04561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.395537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501400c-b957-48e4-b5bd-813cac04561a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.395823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501400c-b957-48e4-b5bd-813cac04561a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.396102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501400c-b957-48e4-b5bd-813cac04561a None admin] PATH_INFO: `/v3/users/efa89e576bc74765b3b490cd3db23bae` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.397301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501400c-b957-48e4-b5bd-813cac04561a None admin] RBAC: Authorizing `identity:get_user(user_id=efa89e576bc74765b3b490cd3db23bae)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:19.398752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501400c-b957-48e4-b5bd-813cac04561a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:19.400543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 540/1092] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/users/efa89e576bc74765b3b490cd3db23bae => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:19.406087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501400c-b957-48e4-b5bd-813cac04561a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.425470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a3deb-6e02-4592-a3da-2a9b4536c018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.426124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a3deb-6e02-4592-a3da-2a9b4536c018 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.426405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a3deb-6e02-4592-a3da-2a9b4536c018 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.426676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a3deb-6e02-4592-a3da-2a9b4536c018 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.427449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a3deb-6e02-4592-a3da-2a9b4536c018 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:19.427750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a3deb-6e02-4592-a3da-2a9b4536c018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:19.429107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a3deb-6e02-4592-a3da-2a9b4536c018 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:19.437202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 553/1093] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:19.442937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a3deb-6e02-4592-a3da-2a9b4536c018 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.495631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:19.506383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501400c-b957-48e4-b5bd-813cac04561a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.539525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba596a1-0d4f-4149-8233-308084bbd4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.540191 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba596a1-0d4f-4149-8233-308084bbd4b0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.540471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba596a1-0d4f-4149-8233-308084bbd4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.540753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba596a1-0d4f-4149-8233-308084bbd4b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.543583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a3deb-6e02-4592-a3da-2a9b4536c018 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.576891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533056c-e59f-46f7-be3a-b28671f9a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.577679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533056c-e59f-46f7-be3a-b28671f9a953 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.577961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533056c-e59f-46f7-be3a-b28671f9a953 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.578238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533056c-e59f-46f7-be3a-b28671f9a953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.588649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533056c-e59f-46f7-be3a-b28671f9a953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:19.590085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533056c-e59f-46f7-be3a-b28671f9a953 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:19.606577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba596a1-0d4f-4149-8233-308084bbd4b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-4533056c-e59f-46f7-be3a-b28671f9a953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:38:19.607289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:19.608267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:19.609168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:38:19.609168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:38:19.609168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:19.609168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:19.609409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba596a1-0d4f-4149-8233-308084bbd4b0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:19.609483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 554/1094] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:19.614380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533056c-e59f-46f7-be3a-b28671f9a953 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.630285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:19.695135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:19.705688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 541/1095] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 15:38:19.711490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba596a1-0d4f-4149-8233-308084bbd4b0 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.714411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533056c-e59f-46f7-be3a-b28671f9a953 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.747566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37289e17-a690-464d-96fe-9a59ffda8a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.748217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37289e17-a690-464d-96fe-9a59ffda8a41 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.748502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37289e17-a690-464d-96fe-9a59ffda8a41 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.748781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37289e17-a690-464d-96fe-9a59ffda8a41 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.749494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37289e17-a690-464d-96fe-9a59ffda8a41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:19.749800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37289e17-a690-464d-96fe-9a59ffda8a41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:19.751218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37289e17-a690-464d-96fe-9a59ffda8a41 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:19.762477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 555/1096] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:19.768352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37289e17-a690-464d-96fe-9a59ffda8a41 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.807153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba596a1-0d4f-4149-8233-308084bbd4b0 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.840219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11523c-a3a3-4794-b8d4-0272ca27d310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.841064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11523c-a3a3-4794-b8d4-0272ca27d310 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.841349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11523c-a3a3-4794-b8d4-0272ca27d310 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.841624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11523c-a3a3-4794-b8d4-0272ca27d310 None admin] PATH_INFO: `/v3/projects/d59dea7b3bef496f86bbe3d53591977f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.841738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:19.842800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11523c-a3a3-4794-b8d4-0272ca27d310 None admin] RBAC: Authorizing `identity:get_project(project_id=d59dea7b3bef496f86bbe3d53591977f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:19.844498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11523c-a3a3-4794-b8d4-0272ca27d310 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:19.846719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 542/1097] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/projects/d59dea7b3bef496f86bbe3d53591977f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:19.852559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11523c-a3a3-4794-b8d4-0272ca27d310 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.867637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37289e17-a690-464d-96fe-9a59ffda8a41 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.901192 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc69b9-c0b7-444e-9bb2-e170ca66aedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.901980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc69b9-c0b7-444e-9bb2-e170ca66aedf None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.902263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc69b9-c0b7-444e-9bb2-e170ca66aedf None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.902540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc69b9-c0b7-444e-9bb2-e170ca66aedf None admin] PATH_INFO: `/v3/users/nova` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.908293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:19.910844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc69b9-c0b7-444e-9bb2-e170ca66aedf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:19.912717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc69b9-c0b7-444e-9bb2-e170ca66aedf None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-fddc69b9-c0b7-444e-9bb2-e170ca66aedf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:38:19.926985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:19.927973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:19.929021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 556/1098] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 15:38:19.933749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc69b9-c0b7-444e-9bb2-e170ca66aedf None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:19.949240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11523c-a3a3-4794-b8d4-0272ca27d310 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:19.988163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3593e4-d6a4-4c7c-9815-8b16feeb40a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:19.988821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3593e4-d6a4-4c7c-9815-8b16feeb40a3 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:19.989102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3593e4-d6a4-4c7c-9815-8b16feeb40a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:19.989381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3593e4-d6a4-4c7c-9815-8b16feeb40a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:19.990214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3593e4-d6a4-4c7c-9815-8b16feeb40a3 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=d59dea7b3bef496f86bbe3d53591977f, user.id=efa89e576bc74765b3b490cd3db23bae) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:19.990521 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3593e4-d6a4-4c7c-9815-8b16feeb40a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:19.991986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3593e4-d6a4-4c7c-9815-8b16feeb40a3 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:19.997822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 543/1099] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/role_assignments?user.id=efa89e576bc74765b3b490cd3db23bae&scope.project.id=d59dea7b3bef496f86bbe3d53591977f&role.id=433478cc318a467c931f13a757f161fa => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:20.050470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc69b9-c0b7-444e-9bb2-e170ca66aedf None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:20.086616 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb247ec-e99e-4095-b7c1-db74bb083b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:20.087320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb247ec-e99e-4095-b7c1-db74bb083b01 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:20.087600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb247ec-e99e-4095-b7c1-db74bb083b01 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:20.087876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb247ec-e99e-4095-b7c1-db74bb083b01 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:20.088718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb247ec-e99e-4095-b7c1-db74bb083b01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:20.089027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb247ec-e99e-4095-b7c1-db74bb083b01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:20.090479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb247ec-e99e-4095-b7c1-db74bb083b01 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:20.092435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:20.110220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 557/1100] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Jun 15 15:38:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:20.116582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3593e4-d6a4-4c7c-9815-8b16feeb40a3 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:20.219703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3593e4-d6a4-4c7c-9815-8b16feeb40a3 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:20.241605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:20.254373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6bb98-7bf8-4abd-81b8-be05d60039a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:20.255054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6bb98-7bf8-4abd-81b8-be05d60039a5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:20.255341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6bb98-7bf8-4abd-81b8-be05d60039a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:20.255623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6bb98-7bf8-4abd-81b8-be05d60039a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:20.325549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6bb98-7bf8-4abd-81b8-be05d60039a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:20.327009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6bb98-7bf8-4abd-81b8-be05d60039a5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:20.421634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 544/1101] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:38:20.428129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb247ec-e99e-4095-b7c1-db74bb083b01 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:20.532048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb247ec-e99e-4095-b7c1-db74bb083b01 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:20.565042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a33371-aca8-4258-8c37-72fd1ab0c207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:20.565819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a33371-aca8-4258-8c37-72fd1ab0c207 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:20.566099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a33371-aca8-4258-8c37-72fd1ab0c207 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:20.566372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a33371-aca8-4258-8c37-72fd1ab0c207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:20.576039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a33371-aca8-4258-8c37-72fd1ab0c207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:20.577472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a33371-aca8-4258-8c37-72fd1ab0c207 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-40a33371-aca8-4258-8c37-72fd1ab0c207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:38:20.594715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:20.595787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:20.597657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:38:20.597657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:38:20.597657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:38:20.597657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:20.597657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 558/1102] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:20.601978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6bb98-7bf8-4abd-81b8-be05d60039a5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:20.624541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:20.702477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6bb98-7bf8-4abd-81b8-be05d60039a5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:20.734870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843947d9-2d6c-4461-913b-1a3bc0cbecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:20.735521 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843947d9-2d6c-4461-913b-1a3bc0cbecfb None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:20.735800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843947d9-2d6c-4461-913b-1a3bc0cbecfb None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:20.736074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843947d9-2d6c-4461-913b-1a3bc0cbecfb None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:20.737299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843947d9-2d6c-4461-913b-1a3bc0cbecfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:20.737602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843947d9-2d6c-4461-913b-1a3bc0cbecfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:20.738966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843947d9-2d6c-4461-913b-1a3bc0cbecfb None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:20.749761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 545/1103] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:20.756066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a33371-aca8-4258-8c37-72fd1ab0c207 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:20.801019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:20.853943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a33371-aca8-4258-8c37-72fd1ab0c207 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:20.889268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd794cc-c440-4c8b-8d37-07b3c784d1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:20.890072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd794cc-c440-4c8b-8d37-07b3c784d1b4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:20.890355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd794cc-c440-4c8b-8d37-07b3c784d1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:20.890626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd794cc-c440-4c8b-8d37-07b3c784d1b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:20.897444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd794cc-c440-4c8b-8d37-07b3c784d1b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:20.898908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd794cc-c440-4c8b-8d37-07b3c784d1b4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-afd794cc-c440-4c8b-8d37-07b3c784d1b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:38:20.912252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:20.913299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:20.914646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:38:20.914646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:38:20.914646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:38:20.914646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:38:20.914646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:38:20.914646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:38:20.914646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:38:20.914646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:20.914646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 559/1104] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:20.919182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843947d9-2d6c-4461-913b-1a3bc0cbecfb None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:21.002391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:21.020363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843947d9-2d6c-4461-913b-1a3bc0cbecfb None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:21.061772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2b07f-2ea6-42f8-ae51-7e2a2bcb7910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:21.062354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2b07f-2ea6-42f8-ae51-7e2a2bcb7910 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:21.062638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2b07f-2ea6-42f8-ae51-7e2a2bcb7910 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:21.062946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2b07f-2ea6-42f8-ae51-7e2a2bcb7910 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:21.063727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2b07f-2ea6-42f8-ae51-7e2a2bcb7910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:21.064037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2b07f-2ea6-42f8-ae51-7e2a2bcb7910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:21.065495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2b07f-2ea6-42f8-ae51-7e2a2bcb7910 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:21.077169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 546/1105] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:38:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:21.083196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd794cc-c440-4c8b-8d37-07b3c784d1b4 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:21.183990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd794cc-c440-4c8b-8d37-07b3c784d1b4 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:21.186246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:21.218465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0baf6-50a5-4bd4-bf0e-b176da6975e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:21.219203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0baf6-50a5-4bd4-bf0e-b176da6975e4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:21.219485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0baf6-50a5-4bd4-bf0e-b176da6975e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:21.219760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0baf6-50a5-4bd4-bf0e-b176da6975e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:21.220576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0baf6-50a5-4bd4-bf0e-b176da6975e4 None admin] RBAC: Adding query filter params (role.id=7a222a4a0d3b41b08325b7ad1e9b6322, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=4327e4eed5a1436ea1a00c2209bc5750) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:21.220886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0baf6-50a5-4bd4-bf0e-b176da6975e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:21.222819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0baf6-50a5-4bd4-bf0e-b176da6975e4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:21.228506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 560/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:21 2022] GET /identity/v3/role_assignments?user.id=4327e4eed5a1436ea1a00c2209bc5750&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=7a222a4a0d3b41b08325b7ad1e9b6322 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:22.161774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976a98f-e832-4d94-9b51-bd6c1abcc6de None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:22.162303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976a98f-e832-4d94-9b51-bd6c1abcc6de None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:22.162303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976a98f-e832-4d94-9b51-bd6c1abcc6de None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:22.163733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 547/1107] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:22.168893 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e9edb-9e46-4333-8bac-c9535ccfc6c3 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:22.169303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e9edb-9e46-4333-8bac-c9535ccfc6c3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:22.169446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e9edb-9e46-4333-8bac-c9535ccfc6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:22.186341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-275e9edb-9e46-4333-8bac-c9535ccfc6c3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:22.187577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-275e9edb-9e46-4333-8bac-c9535ccfc6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:22.250898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 561/1108] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:22.275799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976a98f-e832-4d94-9b51-bd6c1abcc6de None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:22.376707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976a98f-e832-4d94-9b51-bd6c1abcc6de None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:22.412586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894e4e8-f520-4877-adb4-cda10bc21b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:22.413293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894e4e8-f520-4877-adb4-cda10bc21b63 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:22.413578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894e4e8-f520-4877-adb4-cda10bc21b63 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:22.413854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894e4e8-f520-4877-adb4-cda10bc21b63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:22.415043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894e4e8-f520-4877-adb4-cda10bc21b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:22.416476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894e4e8-f520-4877-adb4-cda10bc21b63 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:22.418093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 548/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:22.423798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e9edb-9e46-4333-8bac-c9535ccfc6c3 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:22.515662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:22.520116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e9edb-9e46-4333-8bac-c9535ccfc6c3 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:22.552320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148aaf7c-c9ff-4888-b133-fe1cb45ce15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:22.552970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aaf7c-c9ff-4888-b133-fe1cb45ce15b None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:22.553246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aaf7c-c9ff-4888-b133-fe1cb45ce15b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:22.553516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148aaf7c-c9ff-4888-b133-fe1cb45ce15b None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:22.554209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148aaf7c-c9ff-4888-b133-fe1cb45ce15b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:22.555651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148aaf7c-c9ff-4888-b133-fe1cb45ce15b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:22.574581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 562/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:38:22 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:24.687794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f006ac7-ca21-46fc-ab2a-98f0b56f7f8b None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:24.688371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f006ac7-ca21-46fc-ab2a-98f0b56f7f8b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:24.688371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f006ac7-ca21-46fc-ab2a-98f0b56f7f8b None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:24.690241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 549/1111] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:24.695497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9211e-3edd-4b79-84a2-10b78efbca7d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:24.695894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9211e-3edd-4b79-84a2-10b78efbca7d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:24.696057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9211e-3edd-4b79-84a2-10b78efbca7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:24.712826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-c7b9211e-3edd-4b79-84a2-10b78efbca7d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:24.713944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c7b9211e-3edd-4b79-84a2-10b78efbca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:24.780400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 563/1112] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:24.804113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f006ac7-ca21-46fc-ab2a-98f0b56f7f8b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:24.816205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:24.902681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f006ac7-ca21-46fc-ab2a-98f0b56f7f8b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:24.935834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf528f4-f32d-4d99-bbe2-49ddec88dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:24.936670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf528f4-f32d-4d99-bbe2-49ddec88dc67 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:24.936956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf528f4-f32d-4d99-bbe2-49ddec88dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:24.937235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf528f4-f32d-4d99-bbe2-49ddec88dc67 None admin] PATH_INFO: `/v3/roles/7fb980f85eb04293a0d1dba515bf0a2c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:24.938691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf528f4-f32d-4d99-bbe2-49ddec88dc67 None admin] RBAC: Authorizing `identity:get_role(role_id=7fb980f85eb04293a0d1dba515bf0a2c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:24.940098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf528f4-f32d-4d99-bbe2-49ddec88dc67 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:24.941445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 550/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:24 2022] GET /identity/v3/roles/7fb980f85eb04293a0d1dba515bf0a2c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:24.947089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9211e-3edd-4b79-84a2-10b78efbca7d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:25.027485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:25.043653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9211e-3edd-4b79-84a2-10b78efbca7d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:25.076069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76442a4f-f0cd-44f5-904b-27d900a29e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:25.076926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76442a4f-f0cd-44f5-904b-27d900a29e61 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:25.077206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76442a4f-f0cd-44f5-904b-27d900a29e61 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:25.077480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76442a4f-f0cd-44f5-904b-27d900a29e61 None admin] PATH_INFO: `/v3/users/8df4d5d445c040cbb992f150dfe366d2` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:25.090420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76442a4f-f0cd-44f5-904b-27d900a29e61 None admin] RBAC: Authorizing `identity:get_user(user_id=8df4d5d445c040cbb992f150dfe366d2)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:25.091888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76442a4f-f0cd-44f5-904b-27d900a29e61 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:25.093503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 564/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:24 2022] GET /identity/v3/users/8df4d5d445c040cbb992f150dfe366d2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:25.099755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf528f4-f32d-4d99-bbe2-49ddec88dc67 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:25.175721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:25.197386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf528f4-f32d-4d99-bbe2-49ddec88dc67 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:25.235548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03da3-4516-41c5-af98-6a7c7c1fa7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:25.236220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03da3-4516-41c5-af98-6a7c7c1fa7d7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:25.237040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03da3-4516-41c5-af98-6a7c7c1fa7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:25.237323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03da3-4516-41c5-af98-6a7c7c1fa7d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:25.302345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e03da3-4516-41c5-af98-6a7c7c1fa7d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:25.303758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e03da3-4516-41c5-af98-6a7c7c1fa7d7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:25.376777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:25.399404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 551/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:38:25.405324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76442a4f-f0cd-44f5-904b-27d900a29e61 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:25.502418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76442a4f-f0cd-44f5-904b-27d900a29e61 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:25.535444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174f415-4beb-4001-8095-35728370afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:25.536290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174f415-4beb-4001-8095-35728370afde None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:25.536575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174f415-4beb-4001-8095-35728370afde None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:25.536851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174f415-4beb-4001-8095-35728370afde None admin] PATH_INFO: `/v3/projects/d59dea7b3bef496f86bbe3d53591977f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:25.538009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174f415-4beb-4001-8095-35728370afde None admin] RBAC: Authorizing `identity:get_project(project_id=d59dea7b3bef496f86bbe3d53591977f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:25.539463 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174f415-4beb-4001-8095-35728370afde None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:25.541126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 565/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:25 2022] GET /identity/v3/projects/d59dea7b3bef496f86bbe3d53591977f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:25.547309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03da3-4516-41c5-af98-6a7c7c1fa7d7 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:25.641125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03da3-4516-41c5-af98-6a7c7c1fa7d7 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:25.673853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c1de4-a01c-4c71-9400-d3ffbc2fccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:25.674978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c1de4-a01c-4c71-9400-d3ffbc2fccc7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:25.675258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c1de4-a01c-4c71-9400-d3ffbc2fccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:25.675527 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c1de4-a01c-4c71-9400-d3ffbc2fccc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:25.676340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c1de4-a01c-4c71-9400-d3ffbc2fccc7 None admin] RBAC: Adding query filter params (role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=d59dea7b3bef496f86bbe3d53591977f, user.id=8df4d5d445c040cbb992f150dfe366d2) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:25.676647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c1de4-a01c-4c71-9400-d3ffbc2fccc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:25.678033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c1de4-a01c-4c71-9400-d3ffbc2fccc7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:25.683282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 552/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:25 2022] GET /identity/v3/role_assignments?user.id=8df4d5d445c040cbb992f150dfe366d2&scope.project.id=d59dea7b3bef496f86bbe3d53591977f&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:27.741343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221bb40-fa82-4306-b93b-a869fbf86a54 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:27.741838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221bb40-fa82-4306-b93b-a869fbf86a54 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:27.741838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221bb40-fa82-4306-b93b-a869fbf86a54 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:27.743190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 566/1118] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:27.748305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a75c2-3859-4705-8f6b-f626f6082c74 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:27.748723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a75c2-3859-4705-8f6b-f626f6082c74 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:27.748867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a75c2-3859-4705-8f6b-f626f6082c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:27.765112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-4b3a75c2-3859-4705-8f6b-f626f6082c74 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:27.766283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4b3a75c2-3859-4705-8f6b-f626f6082c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:27.801242 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:27.831419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 553/1119] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:27.853690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221bb40-fa82-4306-b93b-a869fbf86a54 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:27.866195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:27.951467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221bb40-fa82-4306-b93b-a869fbf86a54 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:27.983361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e38273-07bf-4bdf-86d7-cb9fa4a7743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:27.984189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e38273-07bf-4bdf-86d7-cb9fa4a7743a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:27.984467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e38273-07bf-4bdf-86d7-cb9fa4a7743a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:27.984737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e38273-07bf-4bdf-86d7-cb9fa4a7743a None admin] PATH_INFO: `/v3/roles/7fb980f85eb04293a0d1dba515bf0a2c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:27.986214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e38273-07bf-4bdf-86d7-cb9fa4a7743a None admin] RBAC: Authorizing `identity:get_role(role_id=7fb980f85eb04293a0d1dba515bf0a2c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:27.988074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e38273-07bf-4bdf-86d7-cb9fa4a7743a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:27.989395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 567/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:27 2022] GET /identity/v3/roles/7fb980f85eb04293a0d1dba515bf0a2c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:27.995443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a75c2-3859-4705-8f6b-f626f6082c74 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:28.089616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a75c2-3859-4705-8f6b-f626f6082c74 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:28.122491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170eb48e-05c0-420f-a286-ce6b5724a595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:28.123370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170eb48e-05c0-420f-a286-ce6b5724a595 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:28.123653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170eb48e-05c0-420f-a286-ce6b5724a595 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:28.123929 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170eb48e-05c0-420f-a286-ce6b5724a595 None admin] PATH_INFO: `/v3/users/8df4d5d445c040cbb992f150dfe366d2` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:28.125099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170eb48e-05c0-420f-a286-ce6b5724a595 None admin] RBAC: Authorizing `identity:get_user(user_id=8df4d5d445c040cbb992f150dfe366d2)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:28.126501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170eb48e-05c0-420f-a286-ce6b5724a595 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:28.128641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 554/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:27 2022] GET /identity/v3/users/8df4d5d445c040cbb992f150dfe366d2 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:28.134025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e38273-07bf-4bdf-86d7-cb9fa4a7743a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:28.208738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:28.229639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e38273-07bf-4bdf-86d7-cb9fa4a7743a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:28.262975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4880e-dcf6-4f76-8dbb-03d7605beecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:28.263753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4880e-dcf6-4f76-8dbb-03d7605beecf None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:28.264035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4880e-dcf6-4f76-8dbb-03d7605beecf None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:28.264310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4880e-dcf6-4f76-8dbb-03d7605beecf None admin] PATH_INFO: `/v3/projects/d59dea7b3bef496f86bbe3d53591977f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:28.265466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4880e-dcf6-4f76-8dbb-03d7605beecf None admin] RBAC: Authorizing `identity:get_project(project_id=d59dea7b3bef496f86bbe3d53591977f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:28.266915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4880e-dcf6-4f76-8dbb-03d7605beecf None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:28.268854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 568/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:28 2022] GET /identity/v3/projects/d59dea7b3bef496f86bbe3d53591977f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:28.274538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170eb48e-05c0-420f-a286-ce6b5724a595 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:28.315248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:28.370198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170eb48e-05c0-420f-a286-ce6b5724a595 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:28.402632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f94da5-ef66-4d3b-8fb6-e6c199bef5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:28.403523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f94da5-ef66-4d3b-8fb6-e6c199bef5af None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:28.403801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f94da5-ef66-4d3b-8fb6-e6c199bef5af None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:28.404070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f94da5-ef66-4d3b-8fb6-e6c199bef5af None admin] PATH_INFO: `/v3/projects/d59dea7b3bef496f86bbe3d53591977f/users/8df4d5d445c040cbb992f150dfe366d2/roles/7fb980f85eb04293a0d1dba515bf0a2c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:28.406204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f94da5-ef66-4d3b-8fb6-e6c199bef5af None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dea7b3bef496f86bbe3d53591977f, user_id=8df4d5d445c040cbb992f150dfe366d2, role_id=7fb980f85eb04293a0d1dba515bf0a2c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:28.407638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f94da5-ef66-4d3b-8fb6-e6c199bef5af None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:28.419805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 555/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:38:28 2022] PUT /identity/v3/projects/d59dea7b3bef496f86bbe3d53591977f/users/8df4d5d445c040cbb992f150dfe366d2/roles/7fb980f85eb04293a0d1dba515bf0a2c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:38:30.479778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8fb5d-1343-4ac7-be1c-cc719a6f7153 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:30.480193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8fb5d-1343-4ac7-be1c-cc719a6f7153 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:30.480338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8fb5d-1343-4ac7-be1c-cc719a6f7153 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:30.481706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 569/1124] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:30.487276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088e6ce-12a4-41a6-ba4e-fd7a99f1e030 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:30.487648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088e6ce-12a4-41a6-ba4e-fd7a99f1e030 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:30.487828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088e6ce-12a4-41a6-ba4e-fd7a99f1e030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:30.505218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-a088e6ce-12a4-41a6-ba4e-fd7a99f1e030 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:30.506382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a088e6ce-12a4-41a6-ba4e-fd7a99f1e030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:30.574138 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 556/1125] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:30.597617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8fb5d-1343-4ac7-be1c-cc719a6f7153 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:30.695760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8fb5d-1343-4ac7-be1c-cc719a6f7153 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:30.727996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025bb5b-7336-4cef-8081-7155903eef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:30.728770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025bb5b-7336-4cef-8081-7155903eef8f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:30.729052 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025bb5b-7336-4cef-8081-7155903eef8f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:30.729328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025bb5b-7336-4cef-8081-7155903eef8f None admin] PATH_INFO: `/v3/roles/7fb980f85eb04293a0d1dba515bf0a2c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:30.730800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025bb5b-7336-4cef-8081-7155903eef8f None admin] RBAC: Authorizing `identity:get_role(role_id=7fb980f85eb04293a0d1dba515bf0a2c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:30.732163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025bb5b-7336-4cef-8081-7155903eef8f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:30.733506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 570/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:30 2022] GET /identity/v3/roles/7fb980f85eb04293a0d1dba515bf0a2c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:30.739588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088e6ce-12a4-41a6-ba4e-fd7a99f1e030 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:30.767839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:30.834496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088e6ce-12a4-41a6-ba4e-fd7a99f1e030 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:30.867872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f290c-5902-435b-b983-8a67a8c305d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:30.868742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f290c-5902-435b-b983-8a67a8c305d6 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:30.869027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f290c-5902-435b-b983-8a67a8c305d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:30.869306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f290c-5902-435b-b983-8a67a8c305d6 None admin] PATH_INFO: `/v3/users/8df4d5d445c040cbb992f150dfe366d2` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:30.870494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f290c-5902-435b-b983-8a67a8c305d6 None admin] RBAC: Authorizing `identity:get_user(user_id=8df4d5d445c040cbb992f150dfe366d2)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:30.871954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f290c-5902-435b-b983-8a67a8c305d6 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:30.873592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 557/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:30 2022] GET /identity/v3/users/8df4d5d445c040cbb992f150dfe366d2 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:30.880105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025bb5b-7336-4cef-8081-7155903eef8f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:30.896588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:30.978632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025bb5b-7336-4cef-8081-7155903eef8f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:31.012456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa928b3-3356-44d1-ace8-2a133af48119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:31.013096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa928b3-3356-44d1-ace8-2a133af48119 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:31.013377 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa928b3-3356-44d1-ace8-2a133af48119 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:31.013652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa928b3-3356-44d1-ace8-2a133af48119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:31.078851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa928b3-3356-44d1-ace8-2a133af48119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:31.080833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa928b3-3356-44d1-ace8-2a133af48119 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:31.094394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:31.174579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 571/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:38:31.180606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f290c-5902-435b-b983-8a67a8c305d6 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:31.277451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:31.281637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f290c-5902-435b-b983-8a67a8c305d6 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:31.314847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff6a68-afef-44b6-908b-4997ca22fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:31.315696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff6a68-afef-44b6-908b-4997ca22fac7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:31.315983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff6a68-afef-44b6-908b-4997ca22fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:31.316261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff6a68-afef-44b6-908b-4997ca22fac7 None admin] PATH_INFO: `/v3/projects/d59dea7b3bef496f86bbe3d53591977f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:31.317427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff6a68-afef-44b6-908b-4997ca22fac7 None admin] RBAC: Authorizing `identity:get_project(project_id=d59dea7b3bef496f86bbe3d53591977f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:31.318880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff6a68-afef-44b6-908b-4997ca22fac7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:31.320566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 558/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:31 2022] GET /identity/v3/projects/d59dea7b3bef496f86bbe3d53591977f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:31.327030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa928b3-3356-44d1-ace8-2a133af48119 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:31.422036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa928b3-3356-44d1-ace8-2a133af48119 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:31.436106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:31.459439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1a33-30c6-479c-bb79-70c77826c8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:31.460089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1a33-30c6-479c-bb79-70c77826c8db None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:31.460370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1a33-30c6-479c-bb79-70c77826c8db None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:31.460645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1a33-30c6-479c-bb79-70c77826c8db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:31.461462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1a33-30c6-479c-bb79-70c77826c8db None admin] RBAC: Adding query filter params (role.id=7fb980f85eb04293a0d1dba515bf0a2c, scope.project.id=d59dea7b3bef496f86bbe3d53591977f, user.id=8df4d5d445c040cbb992f150dfe366d2) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:31.461772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1a33-30c6-479c-bb79-70c77826c8db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:31.463315 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1a33-30c6-479c-bb79-70c77826c8db None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:31.468969 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 572/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:31 2022] GET /identity/v3/role_assignments?user.id=8df4d5d445c040cbb992f150dfe366d2&scope.project.id=d59dea7b3bef496f86bbe3d53591977f&role.id=7fb980f85eb04293a0d1dba515bf0a2c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:33.582049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e728b0-2921-43c7-8e65-4b1ff7fc807e None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:33.582591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e728b0-2921-43c7-8e65-4b1ff7fc807e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:33.582591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e728b0-2921-43c7-8e65-4b1ff7fc807e None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:33.583966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 559/1131] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:33.589131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c021f9a-309b-46c0-86c7-8b32f5ced28e None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:33.589526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c021f9a-309b-46c0-86c7-8b32f5ced28e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:33.589716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c021f9a-309b-46c0-86c7-8b32f5ced28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:33.605889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5c021f9a-309b-46c0-86c7-8b32f5ced28e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:33.607108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5c021f9a-309b-46c0-86c7-8b32f5ced28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:33.671140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 573/1132] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:33.694973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e728b0-2921-43c7-8e65-4b1ff7fc807e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:33.791977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e728b0-2921-43c7-8e65-4b1ff7fc807e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:33.824347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea11a3e-6304-402a-9696-abd9148d8333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:33.825113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea11a3e-6304-402a-9696-abd9148d8333 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:33.825396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea11a3e-6304-402a-9696-abd9148d8333 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:33.825669 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea11a3e-6304-402a-9696-abd9148d8333 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:33.826848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea11a3e-6304-402a-9696-abd9148d8333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:33.828258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea11a3e-6304-402a-9696-abd9148d8333 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:33.829856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 560/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:33.835460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c021f9a-309b-46c0-86c7-8b32f5ced28e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:33.930326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c021f9a-309b-46c0-86c7-8b32f5ced28e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:33.939203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:33.962668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef060e66-e008-4684-a294-2ec0ec30e2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:33.963343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef060e66-e008-4684-a294-2ec0ec30e2c2 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:33.963623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef060e66-e008-4684-a294-2ec0ec30e2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:33.963894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef060e66-e008-4684-a294-2ec0ec30e2c2 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:33.964594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef060e66-e008-4684-a294-2ec0ec30e2c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:33.966008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef060e66-e008-4684-a294-2ec0ec30e2c2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:33.990700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 574/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:38:33 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:36.104316 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deadbfb-41cc-4548-badc-44819033963a None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:36.104844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deadbfb-41cc-4548-badc-44819033963a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:36.104932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deadbfb-41cc-4548-badc-44819033963a None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:36.106233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 561/1135] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:36.111856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb43bf-29f3-4f11-a369-cca772c12d9a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:36.112243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb43bf-29f3-4f11-a369-cca772c12d9a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:36.112410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb43bf-29f3-4f11-a369-cca772c12d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:36.128546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5cbb43bf-29f3-4f11-a369-cca772c12d9a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:36.129718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5cbb43bf-29f3-4f11-a369-cca772c12d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:36.193565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 575/1136] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:36.220691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deadbfb-41cc-4548-badc-44819033963a None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:36.255602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:36.322439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deadbfb-41cc-4548-badc-44819033963a None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:36.355748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb91d00-ee20-4676-83e6-37a28b0bcf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:36.356465 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb91d00-ee20-4676-83e6-37a28b0bcf51 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:36.356748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb91d00-ee20-4676-83e6-37a28b0bcf51 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:36.357023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb91d00-ee20-4676-83e6-37a28b0bcf51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:36.358173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb91d00-ee20-4676-83e6-37a28b0bcf51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:36.359622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb91d00-ee20-4676-83e6-37a28b0bcf51 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:36.361224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 562/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:38:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:36.366943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb43bf-29f3-4f11-a369-cca772c12d9a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:36.452463 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:36.463324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb43bf-29f3-4f11-a369-cca772c12d9a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:36.496595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff7d64-812e-4f75-966b-8055b4e406b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:36.497256 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff7d64-812e-4f75-966b-8055b4e406b9 None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:36.497541 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff7d64-812e-4f75-966b-8055b4e406b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:36.497820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff7d64-812e-4f75-966b-8055b4e406b9 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:36.498521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff7d64-812e-4f75-966b-8055b4e406b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:36.499985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff7d64-812e-4f75-966b-8055b4e406b9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:36.519342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 576/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:38:36 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:38.664446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877db883-742b-42d7-b812-dea0b5d60479 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:38.664446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877db883-742b-42d7-b812-dea0b5d60479 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:38.664446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877db883-742b-42d7-b812-dea0b5d60479 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:38.665079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 563/1139] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:38.669862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d7082-649c-4cf8-b85a-ca67efda9c00 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:38.670255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d7082-649c-4cf8-b85a-ca67efda9c00 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:38.670423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d7082-649c-4cf8-b85a-ca67efda9c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:38.687787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-fe0d7082-649c-4cf8-b85a-ca67efda9c00 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:38.689547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fe0d7082-649c-4cf8-b85a-ca67efda9c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:38.760736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 577/1140] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:38.784428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877db883-742b-42d7-b812-dea0b5d60479 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:38.886237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877db883-742b-42d7-b812-dea0b5d60479 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:38.887896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:38.922055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cac64c-eb54-47d3-b5d4-137976ba6a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:38.922783 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cac64c-eb54-47d3-b5d4-137976ba6a7b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:38.923686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cac64c-eb54-47d3-b5d4-137976ba6a7b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:38.923961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cac64c-eb54-47d3-b5d4-137976ba6a7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:38.934353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cac64c-eb54-47d3-b5d4-137976ba6a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-b1cac64c-eb54-47d3-b5d4-137976ba6a7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:38.942674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:38.943917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:38.944853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 564/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:38:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:38.950402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d7082-649c-4cf8-b85a-ca67efda9c00 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:39.026833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:39.051788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d7082-649c-4cf8-b85a-ca67efda9c00 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:39.086506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28c9b4-7c92-44c3-a6a7-0650efaf6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:39.087201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28c9b4-7c92-44c3-a6a7-0650efaf6c8d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:39.087486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28c9b4-7c92-44c3-a6a7-0650efaf6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:39.087766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28c9b4-7c92-44c3-a6a7-0650efaf6c8d None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:39.088507 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28c9b4-7c92-44c3-a6a7-0650efaf6c8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:39.088816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28c9b4-7c92-44c3-a6a7-0650efaf6c8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:39.090187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28c9b4-7c92-44c3-a6a7-0650efaf6c8d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:39.098499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 578/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:38:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:39.104555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cac64c-eb54-47d3-b5d4-137976ba6a7b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:39.205807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cac64c-eb54-47d3-b5d4-137976ba6a7b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:39.239950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01231582-e1a0-47f8-9c66-17c87731a33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:39.240819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01231582-e1a0-47f8-9c66-17c87731a33b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:39.241105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01231582-e1a0-47f8-9c66-17c87731a33b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:39.241384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01231582-e1a0-47f8-9c66-17c87731a33b None admin] PATH_INFO: `/v3/users/733745ce598449ca9a82ced1dcecc391` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:39.254284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01231582-e1a0-47f8-9c66-17c87731a33b None admin] RBAC: Authorizing `identity:get_user(user_id=733745ce598449ca9a82ced1dcecc391)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:39.255970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01231582-e1a0-47f8-9c66-17c87731a33b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:39.257660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 565/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:39 2022] GET /identity/v3/users/733745ce598449ca9a82ced1dcecc391 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:39.263509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28c9b4-7c92-44c3-a6a7-0650efaf6c8d None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:39.362486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28c9b4-7c92-44c3-a6a7-0650efaf6c8d None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:39.394908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddbbd5-e0f6-4157-bd8e-331f43d98fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:39.395544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddbbd5-e0f6-4157-bd8e-331f43d98fde None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:39.395822 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddbbd5-e0f6-4157-bd8e-331f43d98fde None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:39.396094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddbbd5-e0f6-4157-bd8e-331f43d98fde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:39.404154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:39.460948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddbbd5-e0f6-4157-bd8e-331f43d98fde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:39.462399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddbbd5-e0f6-4157-bd8e-331f43d98fde None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:39.556776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 579/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:38:39.562580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01231582-e1a0-47f8-9c66-17c87731a33b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:39.584137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:39.659142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01231582-e1a0-47f8-9c66-17c87731a33b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:39.691744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc411f4b-ccd4-4a38-9721-3fdd5a4a7b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:39.692586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc411f4b-ccd4-4a38-9721-3fdd5a4a7b64 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:39.692867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc411f4b-ccd4-4a38-9721-3fdd5a4a7b64 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:39.693142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc411f4b-ccd4-4a38-9721-3fdd5a4a7b64 None admin] PATH_INFO: `/v3/projects/b3cc918f84cd400cbbe910d1b22bc403` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:39.694295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc411f4b-ccd4-4a38-9721-3fdd5a4a7b64 None admin] RBAC: Authorizing `identity:get_project(project_id=b3cc918f84cd400cbbe910d1b22bc403)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:39.695752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc411f4b-ccd4-4a38-9721-3fdd5a4a7b64 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:39.697414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 566/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:39 2022] GET /identity/v3/projects/b3cc918f84cd400cbbe910d1b22bc403 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:39.703381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddbbd5-e0f6-4157-bd8e-331f43d98fde None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:39.742517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:39.802343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddbbd5-e0f6-4157-bd8e-331f43d98fde None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:39.834794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0559c5-4aae-4e42-aace-754939bc493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:39.835436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0559c5-4aae-4e42-aace-754939bc493e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:39.835721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0559c5-4aae-4e42-aace-754939bc493e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:39.835996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0559c5-4aae-4e42-aace-754939bc493e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:39.836812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0559c5-4aae-4e42-aace-754939bc493e None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=b3cc918f84cd400cbbe910d1b22bc403, user.id=733745ce598449ca9a82ced1dcecc391) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:39.837126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0559c5-4aae-4e42-aace-754939bc493e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:39.838550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0559c5-4aae-4e42-aace-754939bc493e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:39.843867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 580/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:39 2022] GET /identity/v3/role_assignments?user.id=733745ce598449ca9a82ced1dcecc391&scope.project.id=b3cc918f84cd400cbbe910d1b22bc403&role.id=433478cc318a467c931f13a757f161fa => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:41.928816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a940a5-fedb-4079-9c2b-14527f210ea7 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:41.929418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a940a5-fedb-4079-9c2b-14527f210ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:41.929418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a940a5-fedb-4079-9c2b-14527f210ea7 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:41.930748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 567/1147] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:41.935949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aebc24-0168-4ec2-a65f-d56f479bdfd4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:41.936326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aebc24-0168-4ec2-a65f-d56f479bdfd4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:41.936511 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aebc24-0168-4ec2-a65f-d56f479bdfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:41.953271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-35aebc24-0168-4ec2-a65f-d56f479bdfd4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:41.954473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-35aebc24-0168-4ec2-a65f-d56f479bdfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:42.019457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 581/1148] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:42.042039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a940a5-fedb-4079-9c2b-14527f210ea7 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:42.129512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:42.143660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a940a5-fedb-4079-9c2b-14527f210ea7 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:42.176210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298caa7-ee5f-4464-bc4c-0d507c7eb577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:42.176981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298caa7-ee5f-4464-bc4c-0d507c7eb577 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:42.177267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298caa7-ee5f-4464-bc4c-0d507c7eb577 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:42.177545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298caa7-ee5f-4464-bc4c-0d507c7eb577 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:42.187695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298caa7-ee5f-4464-bc4c-0d507c7eb577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-f298caa7-ee5f-4464-bc4c-0d507c7eb577 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:42.195778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:42.197021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:42.198122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 568/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:38:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:42.204007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aebc24-0168-4ec2-a65f-d56f479bdfd4 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:42.234951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:42.301818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aebc24-0168-4ec2-a65f-d56f479bdfd4 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:42.334366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec8114-09f2-4e75-9486-9532a166767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:42.335054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec8114-09f2-4e75-9486-9532a166767e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:42.335334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec8114-09f2-4e75-9486-9532a166767e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:42.335610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec8114-09f2-4e75-9486-9532a166767e None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:42.336338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec8114-09f2-4e75-9486-9532a166767e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:42.336643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec8114-09f2-4e75-9486-9532a166767e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:42.338011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec8114-09f2-4e75-9486-9532a166767e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:42.345752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 582/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:38:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:42.351515 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298caa7-ee5f-4464-bc4c-0d507c7eb577 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:42.452917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298caa7-ee5f-4464-bc4c-0d507c7eb577 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:42.485586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916407e3-e2bd-4043-85fb-68f508224cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:42.486512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916407e3-e2bd-4043-85fb-68f508224cd0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:42.486831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916407e3-e2bd-4043-85fb-68f508224cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:42.487112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916407e3-e2bd-4043-85fb-68f508224cd0 None admin] PATH_INFO: `/v3/users/733745ce598449ca9a82ced1dcecc391` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:42.488307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916407e3-e2bd-4043-85fb-68f508224cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=733745ce598449ca9a82ced1dcecc391)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:42.489735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916407e3-e2bd-4043-85fb-68f508224cd0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:42.491412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 569/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:42 2022] GET /identity/v3/users/733745ce598449ca9a82ced1dcecc391 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:42.498087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec8114-09f2-4e75-9486-9532a166767e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:42.590267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:42.597026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec8114-09f2-4e75-9486-9532a166767e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:42.636922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21855d17-7659-469d-b5af-2201753e036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:42.638268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21855d17-7659-469d-b5af-2201753e036a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:42.638795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21855d17-7659-469d-b5af-2201753e036a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:42.639276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21855d17-7659-469d-b5af-2201753e036a None admin] PATH_INFO: `/v3/projects/b3cc918f84cd400cbbe910d1b22bc403` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:42.641266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21855d17-7659-469d-b5af-2201753e036a None admin] RBAC: Authorizing `identity:get_project(project_id=b3cc918f84cd400cbbe910d1b22bc403)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:42.644682 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21855d17-7659-469d-b5af-2201753e036a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:42.647856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 583/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:42 2022] GET /identity/v3/projects/b3cc918f84cd400cbbe910d1b22bc403 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:42.656755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916407e3-e2bd-4043-85fb-68f508224cd0 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:42.680399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:42.759755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916407e3-e2bd-4043-85fb-68f508224cd0 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:42.791636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1356a7c-21f1-4d66-9d37-c5a894382b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:42.792502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1356a7c-21f1-4d66-9d37-c5a894382b1e None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:42.792782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1356a7c-21f1-4d66-9d37-c5a894382b1e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:42.793056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1356a7c-21f1-4d66-9d37-c5a894382b1e None admin] PATH_INFO: `/v3/projects/b3cc918f84cd400cbbe910d1b22bc403/users/733745ce598449ca9a82ced1dcecc391/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:42.794978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1356a7c-21f1-4d66-9d37-c5a894382b1e None admin] RBAC: Authorizing `identity:create_grant(project_id=b3cc918f84cd400cbbe910d1b22bc403, user_id=733745ce598449ca9a82ced1dcecc391, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:42.796394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1356a7c-21f1-4d66-9d37-c5a894382b1e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:42.809952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 570/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:38:42 2022] PUT /identity/v3/projects/b3cc918f84cd400cbbe910d1b22bc403/users/733745ce598449ca9a82ced1dcecc391/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:38:44.909797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce03c68-bb55-4d39-a498-5abf7a46effd None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:44.910349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce03c68-bb55-4d39-a498-5abf7a46effd None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:44.910349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce03c68-bb55-4d39-a498-5abf7a46effd None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:44.911726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 584/1154] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:44.916759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c0aed-fc18-43f0-bb72-3fe65bbe31a0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:44.917130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c0aed-fc18-43f0-bb72-3fe65bbe31a0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:44.917318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c0aed-fc18-43f0-bb72-3fe65bbe31a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:44.934137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-027c0aed-fc18-43f0-bb72-3fe65bbe31a0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:44.935346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-027c0aed-fc18-43f0-bb72-3fe65bbe31a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:45.007323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 571/1155] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:45.030931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce03c68-bb55-4d39-a498-5abf7a46effd None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:45.137199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce03c68-bb55-4d39-a498-5abf7a46effd None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:45.172001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db472f-b5bf-4096-9d86-bb3e33d68d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:45.173321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db472f-b5bf-4096-9d86-bb3e33d68d09 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:45.173598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db472f-b5bf-4096-9d86-bb3e33d68d09 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:45.173866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db472f-b5bf-4096-9d86-bb3e33d68d09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:45.184163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db472f-b5bf-4096-9d86-bb3e33d68d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-48db472f-b5bf-4096-9d86-bb3e33d68d09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:45.192259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:45.193231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:45.194314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 585/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:38:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:45.200357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c0aed-fc18-43f0-bb72-3fe65bbe31a0 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:45.209284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:45.299333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c0aed-fc18-43f0-bb72-3fe65bbe31a0 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:45.332087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e56d2-df9a-4f66-a4f0-853abb1d5da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:45.332750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e56d2-df9a-4f66-a4f0-853abb1d5da1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:45.333036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e56d2-df9a-4f66-a4f0-853abb1d5da1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:45.333314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e56d2-df9a-4f66-a4f0-853abb1d5da1 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:45.334056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e56d2-df9a-4f66-a4f0-853abb1d5da1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:45.334364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e56d2-df9a-4f66-a4f0-853abb1d5da1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:45.335749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e56d2-df9a-4f66-a4f0-853abb1d5da1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:45.343880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 572/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:38:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:38:45.349657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db472f-b5bf-4096-9d86-bb3e33d68d09 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:45.373536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:45.449227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db472f-b5bf-4096-9d86-bb3e33d68d09 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:45.481529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45c71f-6f80-4c15-af03-3293c4807a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:45.482390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45c71f-6f80-4c15-af03-3293c4807a28 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:45.482675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45c71f-6f80-4c15-af03-3293c4807a28 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:45.482983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45c71f-6f80-4c15-af03-3293c4807a28 None admin] PATH_INFO: `/v3/users/733745ce598449ca9a82ced1dcecc391` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:45.484243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45c71f-6f80-4c15-af03-3293c4807a28 None admin] RBAC: Authorizing `identity:get_user(user_id=733745ce598449ca9a82ced1dcecc391)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:45.486269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45c71f-6f80-4c15-af03-3293c4807a28 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:45.487927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 586/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:45 2022] GET /identity/v3/users/733745ce598449ca9a82ced1dcecc391 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:45.493656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e56d2-df9a-4f66-a4f0-853abb1d5da1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:45.559040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:45.592066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e56d2-df9a-4f66-a4f0-853abb1d5da1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:45.625009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce2578-b542-4491-8ac3-5f9a3c16ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:45.625677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce2578-b542-4491-8ac3-5f9a3c16ab9a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:45.625959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce2578-b542-4491-8ac3-5f9a3c16ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:45.626238 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce2578-b542-4491-8ac3-5f9a3c16ab9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:45.690919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce2578-b542-4491-8ac3-5f9a3c16ab9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:45.692346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce2578-b542-4491-8ac3-5f9a3c16ab9a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:45.753309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:45.788006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 573/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:38:45.793855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45c71f-6f80-4c15-af03-3293c4807a28 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:45.882611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:45.893846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45c71f-6f80-4c15-af03-3293c4807a28 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:45.930706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3168d9-b206-4854-9587-cd0bf721aa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:45.931719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3168d9-b206-4854-9587-cd0bf721aa02 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:45.932009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3168d9-b206-4854-9587-cd0bf721aa02 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:45.932288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3168d9-b206-4854-9587-cd0bf721aa02 None admin] PATH_INFO: `/v3/projects/b3cc918f84cd400cbbe910d1b22bc403` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:45.933529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3168d9-b206-4854-9587-cd0bf721aa02 None admin] RBAC: Authorizing `identity:get_project(project_id=b3cc918f84cd400cbbe910d1b22bc403)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:45.936568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3168d9-b206-4854-9587-cd0bf721aa02 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:45.938828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 587/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:45 2022] GET /identity/v3/projects/b3cc918f84cd400cbbe910d1b22bc403 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:45.945628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce2578-b542-4491-8ac3-5f9a3c16ab9a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:46.045015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce2578-b542-4491-8ac3-5f9a3c16ab9a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:46.078479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d2f44-0b34-4a89-8fb3-92f3727abec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:46.079689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2f44-0b34-4a89-8fb3-92f3727abec0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:46.079971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2f44-0b34-4a89-8fb3-92f3727abec0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:46.080249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2f44-0b34-4a89-8fb3-92f3727abec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:46.081082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d2f44-0b34-4a89-8fb3-92f3727abec0 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=b3cc918f84cd400cbbe910d1b22bc403, user.id=733745ce598449ca9a82ced1dcecc391) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:46.081397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d2f44-0b34-4a89-8fb3-92f3727abec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:46.082842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d2f44-0b34-4a89-8fb3-92f3727abec0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:46.088784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 574/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:45 2022] GET /identity/v3/role_assignments?user.id=733745ce598449ca9a82ced1dcecc391&scope.project.id=b3cc918f84cd400cbbe910d1b22bc403&role.id=433478cc318a467c931f13a757f161fa => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:48.257434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370b49-bef7-4411-bcfb-bd628ae2abd8 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:48.257962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370b49-bef7-4411-bcfb-bd628ae2abd8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:48.257962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370b49-bef7-4411-bcfb-bd628ae2abd8 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:48.259379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 588/1162] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:48.264340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c0c4c-5e3d-460b-8068-7d14f7e70063 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:48.264766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c0c4c-5e3d-460b-8068-7d14f7e70063 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:48.264906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c0c4c-5e3d-460b-8068-7d14f7e70063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:48.282166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-073c0c4c-5e3d-460b-8068-7d14f7e70063 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:48.283374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-073c0c4c-5e3d-460b-8068-7d14f7e70063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:48.308457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:48.349435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 575/1163] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:48.372109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24370b49-bef7-4411-bcfb-bd628ae2abd8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:48.470278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24370b49-bef7-4411-bcfb-bd628ae2abd8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:48.504206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe888d-e0e2-4b3c-8fc6-98302b87a736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:48.504842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe888d-e0e2-4b3c-8fc6-98302b87a736 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:48.505127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe888d-e0e2-4b3c-8fc6-98302b87a736 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:48.505404 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe888d-e0e2-4b3c-8fc6-98302b87a736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:48.531576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:48.571782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe888d-e0e2-4b3c-8fc6-98302b87a736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:48.573224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe888d-e0e2-4b3c-8fc6-98302b87a736 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:48.668016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 589/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:38:48.674405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c0c4c-5e3d-460b-8068-7d14f7e70063 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:48.771028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c0c4c-5e3d-460b-8068-7d14f7e70063 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:48.804382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808855d-8550-4bdd-8381-2a634e91494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:48.805120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808855d-8550-4bdd-8381-2a634e91494a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:48.805404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808855d-8550-4bdd-8381-2a634e91494a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:48.805685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808855d-8550-4bdd-8381-2a634e91494a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:48.812218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808855d-8550-4bdd-8381-2a634e91494a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:48.813723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808855d-8550-4bdd-8381-2a634e91494a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-0808855d-8550-4bdd-8381-2a634e91494a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:48.827599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:48.828805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:38:48.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 576/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:38:48 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:48.834624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe888d-e0e2-4b3c-8fc6-98302b87a736 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:48.889717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:48.930254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe888d-e0e2-4b3c-8fc6-98302b87a736 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:48.961846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24a7be-f264-4d82-b88d-d42f4e990a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:48.962490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24a7be-f264-4d82-b88d-d42f4e990a46 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:48.962792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24a7be-f264-4d82-b88d-d42f4e990a46 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:48.963073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24a7be-f264-4d82-b88d-d42f4e990a46 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:48.963773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24a7be-f264-4d82-b88d-d42f4e990a46 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:48.964077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24a7be-f264-4d82-b88d-d42f4e990a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:48.965427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24a7be-f264-4d82-b88d-d42f4e990a46 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:48.972832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 590/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Jun 15 15:38:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:51.047561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3acc73-680b-47f9-900b-c5c315395c16 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:51.048113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3acc73-680b-47f9-900b-c5c315395c16 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:51.048113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3acc73-680b-47f9-900b-c5c315395c16 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:51.049465 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 577/1167] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:51.054472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a90e7-a87b-47b4-814c-b040cba85b6a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:51.054870 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a90e7-a87b-47b4-814c-b040cba85b6a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:51.055055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a90e7-a87b-47b4-814c-b040cba85b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:51.071785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6f6a90e7-a87b-47b4-814c-b040cba85b6a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:51.072945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6f6a90e7-a87b-47b4-814c-b040cba85b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:51.135765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 591/1168] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:51.158649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3acc73-680b-47f9-900b-c5c315395c16 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:51.183583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:51.263211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3acc73-680b-47f9-900b-c5c315395c16 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:51.295801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3167854-4d21-4c92-b2dc-ac8feae4baed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:51.296452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3167854-4d21-4c92-b2dc-ac8feae4baed None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:51.296734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3167854-4d21-4c92-b2dc-ac8feae4baed None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:51.297530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3167854-4d21-4c92-b2dc-ac8feae4baed None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:51.298319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3167854-4d21-4c92-b2dc-ac8feae4baed None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:51.299648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3167854-4d21-4c92-b2dc-ac8feae4baed None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:51.313412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 578/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Wed Jun 15 15:38:51 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:53.448525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ce1b9-9956-4a26-8bd9-596cb0d9c9c3 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:53.449152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ce1b9-9956-4a26-8bd9-596cb0d9c9c3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:53.449152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ce1b9-9956-4a26-8bd9-596cb0d9c9c3 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:53.450431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 592/1170] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:53.455582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26e3d-582a-436f-a454-6d0bad681f77 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:53.456004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26e3d-582a-436f-a454-6d0bad681f77 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:53.456151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26e3d-582a-436f-a454-6d0bad681f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:53.472866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-66e26e3d-582a-436f-a454-6d0bad681f77 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:53.474039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-66e26e3d-582a-436f-a454-6d0bad681f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:53.538749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 579/1171] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:53.562395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ce1b9-9956-4a26-8bd9-596cb0d9c9c3 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:53.615059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:53.661919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ce1b9-9956-4a26-8bd9-596cb0d9c9c3 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:53.693686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9cc11-4af2-4c87-b2d0-dd80fd2032a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:53.694530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9cc11-4af2-4c87-b2d0-dd80fd2032a1 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:53.694836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9cc11-4af2-4c87-b2d0-dd80fd2032a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:53.695112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9cc11-4af2-4c87-b2d0-dd80fd2032a1 None admin] PATH_INFO: `/v3/domains/f9c0f87a3be342e4844df69a806206cd` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:53.705782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9cc11-4af2-4c87-b2d0-dd80fd2032a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f9c0f87a3be342e4844df69a806206cd)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:53.707195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9cc11-4af2-4c87-b2d0-dd80fd2032a1 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:53.708778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 593/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Wed Jun 15 15:38:53 2022] GET /identity/v3/domains/f9c0f87a3be342e4844df69a806206cd => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:53.714517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26e3d-582a-436f-a454-6d0bad681f77 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:53.729045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:53.811798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26e3d-582a-436f-a454-6d0bad681f77 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:53.847335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd97ab-65fb-42d5-9f2e-469b80174008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:53.847984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd97ab-65fb-42d5-9f2e-469b80174008 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:53.848270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd97ab-65fb-42d5-9f2e-469b80174008 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:53.848549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd97ab-65fb-42d5-9f2e-469b80174008 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:53.849255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd97ab-65fb-42d5-9f2e-469b80174008 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:53.850676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd97ab-65fb-42d5-9f2e-469b80174008 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:53.877085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 580/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:38:53 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:56.017394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac7d6a-3af8-4a22-b9e9-af7e85550486 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:56.017928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac7d6a-3af8-4a22-b9e9-af7e85550486 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:56.017928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac7d6a-3af8-4a22-b9e9-af7e85550486 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:56.019334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 594/1174] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:56.024370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c77a72-9dc1-4fcb-8cfc-7cd94eb6565e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:56.024801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c77a72-9dc1-4fcb-8cfc-7cd94eb6565e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:56.024901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c77a72-9dc1-4fcb-8cfc-7cd94eb6565e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:56.041591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-08c77a72-9dc1-4fcb-8cfc-7cd94eb6565e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:56.042769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-08c77a72-9dc1-4fcb-8cfc-7cd94eb6565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:56.098512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:56.108127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 581/1175] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:56.132124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac7d6a-3af8-4a22-b9e9-af7e85550486 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:56.231006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac7d6a-3af8-4a22-b9e9-af7e85550486 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:56.243244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:56.263962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2dc59-3249-4a37-a42c-5fa9eb5bc964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:56.264813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2dc59-3249-4a37-a42c-5fa9eb5bc964 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:56.265090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2dc59-3249-4a37-a42c-5fa9eb5bc964 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:56.265947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2dc59-3249-4a37-a42c-5fa9eb5bc964 None admin] PATH_INFO: `/v3/domains/f9c0f87a3be342e4844df69a806206cd` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:56.267376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2dc59-3249-4a37-a42c-5fa9eb5bc964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f9c0f87a3be342e4844df69a806206cd)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:56.268845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2dc59-3249-4a37-a42c-5fa9eb5bc964 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:56.270367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 595/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Wed Jun 15 15:38:56 2022] GET /identity/v3/domains/f9c0f87a3be342e4844df69a806206cd => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:56.276002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c77a72-9dc1-4fcb-8cfc-7cd94eb6565e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:56.371217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c77a72-9dc1-4fcb-8cfc-7cd94eb6565e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:56.404602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85651b-2436-42c2-a85f-6a442394455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:56.405252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85651b-2436-42c2-a85f-6a442394455f None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:56.405530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85651b-2436-42c2-a85f-6a442394455f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:56.405800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85651b-2436-42c2-a85f-6a442394455f None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:56.407053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85651b-2436-42c2-a85f-6a442394455f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:56.408486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85651b-2436-42c2-a85f-6a442394455f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:56.428122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 582/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:38:56 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:38:58.554258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a2ca-716b-422a-bc63-87c820f663dc None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:58.554818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a2ca-716b-422a-bc63-87c820f663dc None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:58.554818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a2ca-716b-422a-bc63-87c820f663dc None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:58.556187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 596/1178] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:38:58.561215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60480b-54b4-47d7-8e16-c583166447b0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:58.561617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60480b-54b4-47d7-8e16-c583166447b0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:58.561777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60480b-54b4-47d7-8e16-c583166447b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:58.579137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ea60480b-54b4-47d7-8e16-c583166447b0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:38:58.580312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ea60480b-54b4-47d7-8e16-c583166447b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:38:58.623846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:58.645969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 583/1179] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:38:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:38:58.668995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37a2ca-716b-422a-bc63-87c820f663dc None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:58.767436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37a2ca-716b-422a-bc63-87c820f663dc None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:58.800747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84255e1-1818-4ebe-8cb7-dba4cb689d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:58.801526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84255e1-1818-4ebe-8cb7-dba4cb689d16 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:58.801810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84255e1-1818-4ebe-8cb7-dba4cb689d16 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:58.802090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84255e1-1818-4ebe-8cb7-dba4cb689d16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:58.813689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84255e1-1818-4ebe-8cb7-dba4cb689d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-a84255e1-1818-4ebe-8cb7-dba4cb689d16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:38:58.821263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:38:58.822412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:38:58.823380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 597/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:38:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:38:58.828438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60480b-54b4-47d7-8e16-c583166447b0 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:58.923059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60480b-54b4-47d7-8e16-c583166447b0 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:58.955683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b327-002c-43f5-9899-54aa171ca01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:58.956330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b327-002c-43f5-9899-54aa171ca01e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:58.956614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b327-002c-43f5-9899-54aa171ca01e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:58.956892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b327-002c-43f5-9899-54aa171ca01e None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:58.957636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b327-002c-43f5-9899-54aa171ca01e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:58.957945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b327-002c-43f5-9899-54aa171ca01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:58.959333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b327-002c-43f5-9899-54aa171ca01e None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:58.967090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 584/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:38:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:58.972631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84255e1-1818-4ebe-8cb7-dba4cb689d16 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:59.013800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:59.069946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84255e1-1818-4ebe-8cb7-dba4cb689d16 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:59.103228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550328e2-4cc9-45b0-ae03-0d9ef512302a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:59.104084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550328e2-4cc9-45b0-ae03-0d9ef512302a None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:59.104367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550328e2-4cc9-45b0-ae03-0d9ef512302a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:59.104643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550328e2-4cc9-45b0-ae03-0d9ef512302a None admin] PATH_INFO: `/v3/users/a4d043f830ea418da9352ff86c03a2ca` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:59.117180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550328e2-4cc9-45b0-ae03-0d9ef512302a None admin] RBAC: Authorizing `identity:get_user(user_id=a4d043f830ea418da9352ff86c03a2ca)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:59.118615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550328e2-4cc9-45b0-ae03-0d9ef512302a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:59.120246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 598/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:38:58 2022] GET /identity/v3/users/a4d043f830ea418da9352ff86c03a2ca => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:59.125831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b327-002c-43f5-9899-54aa171ca01e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:59.168246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:59.225424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b327-002c-43f5-9899-54aa171ca01e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:59.258641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e8c18-4851-4ce3-8901-f54071ffb31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:59.259310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e8c18-4851-4ce3-8901-f54071ffb31f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:59.259595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e8c18-4851-4ce3-8901-f54071ffb31f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:59.259876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e8c18-4851-4ce3-8901-f54071ffb31f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:59.324889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e8c18-4851-4ce3-8901-f54071ffb31f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:59.326263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e8c18-4851-4ce3-8901-f54071ffb31f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:59.365721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:59.421654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 585/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:38:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:38:59.427570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550328e2-4cc9-45b0-ae03-0d9ef512302a None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:59.521113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:38:59.524999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550328e2-4cc9-45b0-ae03-0d9ef512302a None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:59.557666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708b89d-adbb-4c83-b2bc-f0df947888a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:59.558508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708b89d-adbb-4c83-b2bc-f0df947888a8 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:59.558824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708b89d-adbb-4c83-b2bc-f0df947888a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:59.559100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708b89d-adbb-4c83-b2bc-f0df947888a8 None admin] PATH_INFO: `/v3/projects/33909a7e9b7b4116b0a8ad85e8a5a3ee` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:59.560261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708b89d-adbb-4c83-b2bc-f0df947888a8 None admin] RBAC: Authorizing `identity:get_project(project_id=33909a7e9b7b4116b0a8ad85e8a5a3ee)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:59.561685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708b89d-adbb-4c83-b2bc-f0df947888a8 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:59.563441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 599/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:38:59 2022] GET /identity/v3/projects/33909a7e9b7b4116b0a8ad85e8a5a3ee => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:38:59.569484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e8c18-4851-4ce3-8901-f54071ffb31f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:38:59.665182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e8c18-4851-4ce3-8901-f54071ffb31f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:38:59.697837 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c3448-f7d9-4c33-ab6f-a3b584c7ba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:38:59.698471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c3448-f7d9-4c33-ab6f-a3b584c7ba9f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:38:59.698776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c3448-f7d9-4c33-ab6f-a3b584c7ba9f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:38:59.699055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c3448-f7d9-4c33-ab6f-a3b584c7ba9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:38:59.699872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c3448-f7d9-4c33-ab6f-a3b584c7ba9f None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=33909a7e9b7b4116b0a8ad85e8a5a3ee, user.id=a4d043f830ea418da9352ff86c03a2ca) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:38:59.700182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c3448-f7d9-4c33-ab6f-a3b584c7ba9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:38:59.701601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c3448-f7d9-4c33-ab6f-a3b584c7ba9f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:38:59.707390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 586/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:38:59 2022] GET /identity/v3/role_assignments?user.id=a4d043f830ea418da9352ff86c03a2ca&scope.project.id=33909a7e9b7b4116b0a8ad85e8a5a3ee&role.id=433478cc318a467c931f13a757f161fa => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:39:01.781287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a30d8a-749e-4664-87f7-c4279ac22ece None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:01.781782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a30d8a-749e-4664-87f7-c4279ac22ece None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:01.781782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a30d8a-749e-4664-87f7-c4279ac22ece None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:01.783139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 600/1186] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:39:01.788212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb9c83-1099-46be-998d-56d1b2b41e4c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:01.788655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb9c83-1099-46be-998d-56d1b2b41e4c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:01.788756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb9c83-1099-46be-998d-56d1b2b41e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:01.800429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:01.806120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c1fb9c83-1099-46be-998d-56d1b2b41e4c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:39:01.807326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c1fb9c83-1099-46be-998d-56d1b2b41e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:39:01.871291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 587/1187] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:39:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:39:01.894387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a30d8a-749e-4664-87f7-c4279ac22ece None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:01.994461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a30d8a-749e-4664-87f7-c4279ac22ece None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:02.027306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7d393-fe18-4cbf-86f4-09fe7a6810db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:02.028087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7d393-fe18-4cbf-86f4-09fe7a6810db None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:02.028373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7d393-fe18-4cbf-86f4-09fe7a6810db None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:02.028650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7d393-fe18-4cbf-86f4-09fe7a6810db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:02.039078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7d393-fe18-4cbf-86f4-09fe7a6810db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-d4f7d393-fe18-4cbf-86f4-09fe7a6810db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:39:02.046617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:39:02.047787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:39:02.048695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 601/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:39:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:39:02.054004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb9c83-1099-46be-998d-56d1b2b41e4c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:02.148825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb9c83-1099-46be-998d-56d1b2b41e4c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:02.180992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a86929-e974-4e58-b015-c0fda5b46004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:02.181638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a86929-e974-4e58-b015-c0fda5b46004 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:02.181922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a86929-e974-4e58-b015-c0fda5b46004 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:02.182202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a86929-e974-4e58-b015-c0fda5b46004 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:02.182965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a86929-e974-4e58-b015-c0fda5b46004 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:39:02.183272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a86929-e974-4e58-b015-c0fda5b46004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:39:02.184622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a86929-e974-4e58-b015-c0fda5b46004 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:39:02.186740 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:02.193628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 588/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:39:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:39:02.199515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7d393-fe18-4cbf-86f4-09fe7a6810db None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:02.227303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:02.297894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7d393-fe18-4cbf-86f4-09fe7a6810db None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:02.331191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a97f5-b1d3-4430-b52f-84731400ff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:02.332050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a97f5-b1d3-4430-b52f-84731400ff90 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:02.332329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a97f5-b1d3-4430-b52f-84731400ff90 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:02.332601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a97f5-b1d3-4430-b52f-84731400ff90 None admin] PATH_INFO: `/v3/users/a4d043f830ea418da9352ff86c03a2ca` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:02.333768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a97f5-b1d3-4430-b52f-84731400ff90 None admin] RBAC: Authorizing `identity:get_user(user_id=a4d043f830ea418da9352ff86c03a2ca)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:39:02.335234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a97f5-b1d3-4430-b52f-84731400ff90 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:39:02.336872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 602/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:39:02 2022] GET /identity/v3/users/a4d043f830ea418da9352ff86c03a2ca => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:39:02.342664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a86929-e974-4e58-b015-c0fda5b46004 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:02.440137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a86929-e974-4e58-b015-c0fda5b46004 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:02.472946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a087d9-5d45-43c4-9527-f48d7171ef21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:02.473790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a087d9-5d45-43c4-9527-f48d7171ef21 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:02.474071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a087d9-5d45-43c4-9527-f48d7171ef21 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:02.474344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a087d9-5d45-43c4-9527-f48d7171ef21 None admin] PATH_INFO: `/v3/projects/33909a7e9b7b4116b0a8ad85e8a5a3ee` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:02.475530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a087d9-5d45-43c4-9527-f48d7171ef21 None admin] RBAC: Authorizing `identity:get_project(project_id=33909a7e9b7b4116b0a8ad85e8a5a3ee)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:39:02.476940 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a087d9-5d45-43c4-9527-f48d7171ef21 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:39:02.478620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 589/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:39:02 2022] GET /identity/v3/projects/33909a7e9b7b4116b0a8ad85e8a5a3ee => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:39:02.484517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a97f5-b1d3-4430-b52f-84731400ff90 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:02.574501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:02.581130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a97f5-b1d3-4430-b52f-84731400ff90 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:02.614095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9defa8-f9bb-43f1-9692-efd04ea8ead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:02.614996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9defa8-f9bb-43f1-9692-efd04ea8ead7 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:02.615284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9defa8-f9bb-43f1-9692-efd04ea8ead7 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:02.615563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9defa8-f9bb-43f1-9692-efd04ea8ead7 None admin] PATH_INFO: `/v3/projects/33909a7e9b7b4116b0a8ad85e8a5a3ee/users/a4d043f830ea418da9352ff86c03a2ca/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:02.617510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9defa8-f9bb-43f1-9692-efd04ea8ead7 None admin] RBAC: Authorizing `identity:create_grant(project_id=33909a7e9b7b4116b0a8ad85e8a5a3ee, user_id=a4d043f830ea418da9352ff86c03a2ca, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:39:02.618982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9defa8-f9bb-43f1-9692-efd04ea8ead7 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:39:02.631778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 603/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:39:02 2022] PUT /identity/v3/projects/33909a7e9b7b4116b0a8ad85e8a5a3ee/users/a4d043f830ea418da9352ff86c03a2ca/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:39:04.700494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479e755-ae86-48bb-9b57-70973cd677c8 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:04.700999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479e755-ae86-48bb-9b57-70973cd677c8 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:04.700999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479e755-ae86-48bb-9b57-70973cd677c8 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:04.702936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 590/1193] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:39:04.708116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d45fd-5de4-4988-bf18-03bd4d568241 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:04.708531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d45fd-5de4-4988-bf18-03bd4d568241 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:04.708674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d45fd-5de4-4988-bf18-03bd4d568241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:04.725693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-f94d45fd-5de4-4988-bf18-03bd4d568241 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:39:04.726906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f94d45fd-5de4-4988-bf18-03bd4d568241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:39:04.791452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 604/1194] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:39:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:39:04.818275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479e755-ae86-48bb-9b57-70973cd677c8 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:04.882424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:04.918080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479e755-ae86-48bb-9b57-70973cd677c8 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:04.950113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf30c39-6424-4102-a00d-fc9ecea18642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:04.950962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf30c39-6424-4102-a00d-fc9ecea18642 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:04.951198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf30c39-6424-4102-a00d-fc9ecea18642 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:04.951472 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf30c39-6424-4102-a00d-fc9ecea18642 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:04.961748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf30c39-6424-4102-a00d-fc9ecea18642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-faf30c39-6424-4102-a00d-fc9ecea18642 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:39:04.969353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:39:04.970427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:39:04.971641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 591/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:39:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:39:04.976506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d45fd-5de4-4988-bf18-03bd4d568241 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:05.056590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:05.072823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d45fd-5de4-4988-bf18-03bd4d568241 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:05.104672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a534-0ced-4627-854e-b0b2cc63032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:05.105321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a534-0ced-4627-854e-b0b2cc63032c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:05.105603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a534-0ced-4627-854e-b0b2cc63032c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:05.105878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a534-0ced-4627-854e-b0b2cc63032c None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:05.106613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392a534-0ced-4627-854e-b0b2cc63032c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:39:05.106946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392a534-0ced-4627-854e-b0b2cc63032c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:39:05.108314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392a534-0ced-4627-854e-b0b2cc63032c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:39:05.116580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 605/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:39:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:39:05.122228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf30c39-6424-4102-a00d-fc9ecea18642 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:05.219970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf30c39-6424-4102-a00d-fc9ecea18642 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:05.244290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:05.254714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f9389-16c8-4296-b5db-5c1b3f8f9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:05.255683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f9389-16c8-4296-b5db-5c1b3f8f9cf1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:05.256162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f9389-16c8-4296-b5db-5c1b3f8f9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:05.256664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f9389-16c8-4296-b5db-5c1b3f8f9cf1 None admin] PATH_INFO: `/v3/users/a4d043f830ea418da9352ff86c03a2ca` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:05.258397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f9389-16c8-4296-b5db-5c1b3f8f9cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=a4d043f830ea418da9352ff86c03a2ca)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:39:05.261019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f9389-16c8-4296-b5db-5c1b3f8f9cf1 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:39:05.264118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 592/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Jun 15 15:39:05 2022] GET /identity/v3/users/a4d043f830ea418da9352ff86c03a2ca => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:39:05.270384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a534-0ced-4627-854e-b0b2cc63032c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:05.367137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a534-0ced-4627-854e-b0b2cc63032c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:05.398597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6c1bf-dc6b-438a-9e17-18484c6d152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:05.399252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6c1bf-dc6b-438a-9e17-18484c6d152f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:05.399532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6c1bf-dc6b-438a-9e17-18484c6d152f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:05.399805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6c1bf-dc6b-438a-9e17-18484c6d152f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:05.414602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:05.467691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6c1bf-dc6b-438a-9e17-18484c6d152f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:39:05.469618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6c1bf-dc6b-438a-9e17-18484c6d152f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:39:05.563387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 606/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:39:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 15:39:05.569141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f9389-16c8-4296-b5db-5c1b3f8f9cf1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:05.666067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f9389-16c8-4296-b5db-5c1b3f8f9cf1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:05.698992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46dd35-15db-4ae9-9d46-b104b00c32d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:05.699840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46dd35-15db-4ae9-9d46-b104b00c32d5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:05.700123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46dd35-15db-4ae9-9d46-b104b00c32d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:05.700402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46dd35-15db-4ae9-9d46-b104b00c32d5 None admin] PATH_INFO: `/v3/projects/33909a7e9b7b4116b0a8ad85e8a5a3ee` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:05.701547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46dd35-15db-4ae9-9d46-b104b00c32d5 None admin] RBAC: Authorizing `identity:get_project(project_id=33909a7e9b7b4116b0a8ad85e8a5a3ee)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:39:05.703036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46dd35-15db-4ae9-9d46-b104b00c32d5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:39:05.704718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 593/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Jun 15 15:39:05 2022] GET /identity/v3/projects/33909a7e9b7b4116b0a8ad85e8a5a3ee => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:39:05.710812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6c1bf-dc6b-438a-9e17-18484c6d152f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:39:05.755832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:05.807364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6c1bf-dc6b-438a-9e17-18484c6d152f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:39:05.844273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015e9ed-40a0-4728-a7ef-8789ce6f50ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:39:05.844850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015e9ed-40a0-4728-a7ef-8789ce6f50ed None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:39:05.845131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015e9ed-40a0-4728-a7ef-8789ce6f50ed None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:39:05.845406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015e9ed-40a0-4728-a7ef-8789ce6f50ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:39:05.846254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015e9ed-40a0-4728-a7ef-8789ce6f50ed None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=33909a7e9b7b4116b0a8ad85e8a5a3ee, user.id=a4d043f830ea418da9352ff86c03a2ca) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:39:05.846565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015e9ed-40a0-4728-a7ef-8789ce6f50ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:39:05.848025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015e9ed-40a0-4728-a7ef-8789ce6f50ed None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:39:05.853693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 607/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:39:05 2022] GET /identity/v3/role_assignments?user.id=a4d043f830ea418da9352ff86c03a2ca&scope.project.id=33909a7e9b7b4116b0a8ad85e8a5a3ee&role.id=433478cc318a467c931f13a757f161fa => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:39:15.694940 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=69 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:39:15.851242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=45 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:01.828019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac3fcc-8d6b-43d3-909b-f39891d8d6e4 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:01.828584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac3fcc-8d6b-43d3-909b-f39891d8d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:01.828584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac3fcc-8d6b-43d3-909b-f39891d8d6e4 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:01.829864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 594/1201] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:01.837587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc18136-8768-4d80-8c3c-25a213dc3f46 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:01.837587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc18136-8768-4d80-8c3c-25a213dc3f46 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:01.837587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc18136-8768-4d80-8c3c-25a213dc3f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:01.856401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9cc18136-8768-4d80-8c3c-25a213dc3f46 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:01.858064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9cc18136-8768-4d80-8c3c-25a213dc3f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:01.947967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 608/1202] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:01.975448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac3fcc-8d6b-43d3-909b-f39891d8d6e4 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:02.117432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac3fcc-8d6b-43d3-909b-f39891d8d6e4 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:02.168679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4066e-36ea-433e-8e2f-489942737bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:02.169458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4066e-36ea-433e-8e2f-489942737bf8 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:02.169787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4066e-36ea-433e-8e2f-489942737bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:02.170066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4066e-36ea-433e-8e2f-489942737bf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:02.182395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4066e-36ea-433e-8e2f-489942737bf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:02.184421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4066e-36ea-433e-8e2f-489942737bf8 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-c8b4066e-36ea-433e-8e2f-489942737bf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:02.204534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:02.206270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:02.207952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:02.207952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:02.207952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:02.207952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:02.207952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 595/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:02.212497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc18136-8768-4d80-8c3c-25a213dc3f46 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:02.348662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc18136-8768-4d80-8c3c-25a213dc3f46 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:02.386148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:02.393735 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363712b1-65a7-45ab-885f-3f20f49572b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:02.394379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363712b1-65a7-45ab-885f-3f20f49572b4 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:02.394796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363712b1-65a7-45ab-885f-3f20f49572b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:02.394992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363712b1-65a7-45ab-885f-3f20f49572b4 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:02.395727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363712b1-65a7-45ab-885f-3f20f49572b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:02.396038 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363712b1-65a7-45ab-885f-3f20f49572b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:02.397493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363712b1-65a7-45ab-885f-3f20f49572b4 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:02.411772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 609/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:02.417955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4066e-36ea-433e-8e2f-489942737bf8 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:02.490591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:02.548381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4066e-36ea-433e-8e2f-489942737bf8 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:02.589353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7437e-c6a9-4d1e-b9d4-91eae815e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:02.590039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7437e-c6a9-4d1e-b9d4-91eae815e822 None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:02.590325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7437e-c6a9-4d1e-b9d4-91eae815e822 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:02.590601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7437e-c6a9-4d1e-b9d4-91eae815e822 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:02.591343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7437e-c6a9-4d1e-b9d4-91eae815e822 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:02.592896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7437e-c6a9-4d1e-b9d4-91eae815e822 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:02.621743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 596/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Jun 15 15:40:02 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:40:04.980467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f33a8-a520-4add-a786-79485277329f None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:04.981048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f33a8-a520-4add-a786-79485277329f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:04.981048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f33a8-a520-4add-a786-79485277329f None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:04.982394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 610/1206] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:04.988124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24943974-8b6e-4a0f-97d0-e81c52ebfafc None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:04.988606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24943974-8b6e-4a0f-97d0-e81c52ebfafc None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:04.988727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24943974-8b6e-4a0f-97d0-e81c52ebfafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:05.007984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-24943974-8b6e-4a0f-97d0-e81c52ebfafc None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:05.009316 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-24943974-8b6e-4a0f-97d0-e81c52ebfafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:05.085987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 597/1207] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:05.113009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f33a8-a520-4add-a786-79485277329f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:05.237722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f33a8-a520-4add-a786-79485277329f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:05.277714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42708bf6-7fe7-415e-b5bb-7d1263aad9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:05.278579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42708bf6-7fe7-415e-b5bb-7d1263aad9fc None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:05.278938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42708bf6-7fe7-415e-b5bb-7d1263aad9fc None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:05.279259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42708bf6-7fe7-415e-b5bb-7d1263aad9fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:05.291500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42708bf6-7fe7-415e-b5bb-7d1263aad9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-42708bf6-7fe7-415e-b5bb-7d1263aad9fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:40:05.300785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:05.302231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:05.303150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 611/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:40:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:05.308448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24943974-8b6e-4a0f-97d0-e81c52ebfafc None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:05.350142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:05.426588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24943974-8b6e-4a0f-97d0-e81c52ebfafc None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:05.466479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca777237-20a9-4cf3-a34e-230a3d207597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:05.467252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777237-20a9-4cf3-a34e-230a3d207597 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:05.467581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777237-20a9-4cf3-a34e-230a3d207597 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:05.467932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777237-20a9-4cf3-a34e-230a3d207597 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:05.468795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca777237-20a9-4cf3-a34e-230a3d207597 None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:05.469152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca777237-20a9-4cf3-a34e-230a3d207597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:05.470640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca777237-20a9-4cf3-a34e-230a3d207597 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:05.480022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 598/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:40:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:05.486027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42708bf6-7fe7-415e-b5bb-7d1263aad9fc None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:05.541194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:05.606178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42708bf6-7fe7-415e-b5bb-7d1263aad9fc None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:05.651545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf8f2e-b07b-43f9-9692-a7593e9b6cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:05.652411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf8f2e-b07b-43f9-9692-a7593e9b6cb5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:05.652694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf8f2e-b07b-43f9-9692-a7593e9b6cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:05.653047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf8f2e-b07b-43f9-9692-a7593e9b6cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:05.671370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf8f2e-b07b-43f9-9692-a7593e9b6cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:05.672833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf8f2e-b07b-43f9-9692-a7593e9b6cb5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-a1cf8f2e-b07b-43f9-9692-a7593e9b6cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:05.694021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:05.695221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:05.700882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:05.700882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:05.700882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:05.700882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:05.700882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 612/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:05.701234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca777237-20a9-4cf3-a34e-230a3d207597 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:05.817985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:05.821781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca777237-20a9-4cf3-a34e-230a3d207597 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:05.859617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be23cf3-c56b-41c2-97ae-91b110c5f894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:05.860280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be23cf3-c56b-41c2-97ae-91b110c5f894 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:05.860561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be23cf3-c56b-41c2-97ae-91b110c5f894 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:05.860844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be23cf3-c56b-41c2-97ae-91b110c5f894 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:05.861570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be23cf3-c56b-41c2-97ae-91b110c5f894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:05.861881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be23cf3-c56b-41c2-97ae-91b110c5f894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:05.863294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be23cf3-c56b-41c2-97ae-91b110c5f894 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:05.876268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 599/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:40:05.882406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf8f2e-b07b-43f9-9692-a7593e9b6cb5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:06.000981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf8f2e-b07b-43f9-9692-a7593e9b6cb5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:06.002627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:06.041943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7438ed-7d79-4188-9856-5e73c7d82997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:06.042805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7438ed-7d79-4188-9856-5e73c7d82997 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:06.044245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7438ed-7d79-4188-9856-5e73c7d82997 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:06.044245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7438ed-7d79-4188-9856-5e73c7d82997 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:06.052758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7438ed-7d79-4188-9856-5e73c7d82997 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:06.054291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7438ed-7d79-4188-9856-5e73c7d82997 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-5f7438ed-7d79-4188-9856-5e73c7d82997 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:40:06.070872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:06.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:06.074305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:06.078777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 613/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:06.079725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be23cf3-c56b-41c2-97ae-91b110c5f894 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:06.199334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be23cf3-c56b-41c2-97ae-91b110c5f894 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:06.240451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20462c1e-fb53-43c6-974d-069207a153cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:06.241266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20462c1e-fb53-43c6-974d-069207a153cb None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:06.241556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20462c1e-fb53-43c6-974d-069207a153cb None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:06.241889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20462c1e-fb53-43c6-974d-069207a153cb None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:06.242847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20462c1e-fb53-43c6-974d-069207a153cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:06.243231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20462c1e-fb53-43c6-974d-069207a153cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:06.245314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20462c1e-fb53-43c6-974d-069207a153cb None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:06.265658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 600/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Jun 15 15:40:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:06.271546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7438ed-7d79-4188-9856-5e73c7d82997 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:06.389615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7438ed-7d79-4188-9856-5e73c7d82997 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:06.430984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae40db-672b-4d62-81fd-95f862b413f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:06.431667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae40db-672b-4d62-81fd-95f862b413f6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:06.432030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae40db-672b-4d62-81fd-95f862b413f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:06.432337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae40db-672b-4d62-81fd-95f862b413f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:06.471159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:06.517989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae40db-672b-4d62-81fd-95f862b413f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:06.519592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae40db-672b-4d62-81fd-95f862b413f6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:06.635943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 614/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:40:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:40:06.638288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20462c1e-fb53-43c6-974d-069207a153cb None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:06.661097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:06.751907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20462c1e-fb53-43c6-974d-069207a153cb None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:06.793968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca871b7-3462-4560-b9f1-9660ca67ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:06.794937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca871b7-3462-4560-b9f1-9660ca67ba23 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:06.795280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca871b7-3462-4560-b9f1-9660ca67ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:06.795646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca871b7-3462-4560-b9f1-9660ca67ba23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:06.810013 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca871b7-3462-4560-b9f1-9660ca67ba23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:06.811591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca871b7-3462-4560-b9f1-9660ca67ba23 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-9ca871b7-3462-4560-b9f1-9660ca67ba23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:06.831892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:06.833081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:06.834212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:06.834212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:06.834212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:06.834212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:06.837934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 601/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:06.839826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae40db-672b-4d62-81fd-95f862b413f6 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:06.913501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:06.957110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae40db-672b-4d62-81fd-95f862b413f6 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:06.996005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5171b0-48d8-408e-96a6-4ec0a86bd5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:06.996672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5171b0-48d8-408e-96a6-4ec0a86bd5af None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:06.996951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5171b0-48d8-408e-96a6-4ec0a86bd5af None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:06.997222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5171b0-48d8-408e-96a6-4ec0a86bd5af None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:06.997933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5171b0-48d8-408e-96a6-4ec0a86bd5af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:06.998233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5171b0-48d8-408e-96a6-4ec0a86bd5af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:06.999625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5171b0-48d8-408e-96a6-4ec0a86bd5af None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:07.011628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 615/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:40:07.018304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca871b7-3462-4560-b9f1-9660ca67ba23 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:07.093181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:07.124657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca871b7-3462-4560-b9f1-9660ca67ba23 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:07.169190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486746c-9da9-4e49-9bca-d84d91c8743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:07.169190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486746c-9da9-4e49-9bca-d84d91c8743d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:07.169190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486746c-9da9-4e49-9bca-d84d91c8743d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:07.169190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486746c-9da9-4e49-9bca-d84d91c8743d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:07.178185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1486746c-9da9-4e49-9bca-d84d91c8743d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:07.180584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1486746c-9da9-4e49-9bca-d84d91c8743d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-1486746c-9da9-4e49-9bca-d84d91c8743d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:40:07.196928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:07.198885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:07.199839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:40:07.199839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:40:07.199839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:40:07.199839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:40:07.199839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:40:07.199839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:40:07.199839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:07.199839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:07.199839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 602/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:40:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:07.205314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5171b0-48d8-408e-96a6-4ec0a86bd5af None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:07.324852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5171b0-48d8-408e-96a6-4ec0a86bd5af None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:07.352249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:07.368448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b49910-9b30-4569-adb5-cea7bec1fcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:07.370328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b49910-9b30-4569-adb5-cea7bec1fcce None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:07.370836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b49910-9b30-4569-adb5-cea7bec1fcce None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:07.371267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b49910-9b30-4569-adb5-cea7bec1fcce None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:07.372418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b49910-9b30-4569-adb5-cea7bec1fcce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:07.372933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b49910-9b30-4569-adb5-cea7bec1fcce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:07.374810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b49910-9b30-4569-adb5-cea7bec1fcce None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:07.389227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 616/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:40:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:07.395440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486746c-9da9-4e49-9bca-d84d91c8743d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:07.513057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486746c-9da9-4e49-9bca-d84d91c8743d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:07.548452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:07.555445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9e1ba-1009-4ffd-b9bf-aa4ebdd967f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:07.556418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9e1ba-1009-4ffd-b9bf-aa4ebdd967f0 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:07.557154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9e1ba-1009-4ffd-b9bf-aa4ebdd967f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:07.557541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9e1ba-1009-4ffd-b9bf-aa4ebdd967f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:07.559314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9e1ba-1009-4ffd-b9bf-aa4ebdd967f0 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=a786a95c48c348dbba269e0e33265271) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:07.559731 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9e1ba-1009-4ffd-b9bf-aa4ebdd967f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:07.561397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9e1ba-1009-4ffd-b9bf-aa4ebdd967f0 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:07.567924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 603/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:40:07 2022] GET /identity/v3/role_assignments?user.id=a786a95c48c348dbba269e0e33265271&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:09.818720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99d205-257e-4616-b642-c52d37c267bb None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:09.819376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99d205-257e-4616-b642-c52d37c267bb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:09.819376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99d205-257e-4616-b642-c52d37c267bb None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:09.821067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 617/1220] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:09.826624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8104a4-020e-4380-a467-f3ca85d8e17f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:09.827370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8104a4-020e-4380-a467-f3ca85d8e17f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:09.827370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8104a4-020e-4380-a467-f3ca85d8e17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:09.846348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-2b8104a4-020e-4380-a467-f3ca85d8e17f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:09.847566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2b8104a4-020e-4380-a467-f3ca85d8e17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:09.926088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 604/1221] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 15:40:09.951528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99d205-257e-4616-b642-c52d37c267bb None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:10.060266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99d205-257e-4616-b642-c52d37c267bb None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:10.098485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6711362-8447-40d7-89ff-073a33278787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:10.099289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6711362-8447-40d7-89ff-073a33278787 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:10.099543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6711362-8447-40d7-89ff-073a33278787 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:10.099819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6711362-8447-40d7-89ff-073a33278787 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:10.111515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6711362-8447-40d7-89ff-073a33278787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-b6711362-8447-40d7-89ff-073a33278787 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:40:10.119121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:10.120903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:10.121981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 618/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:40:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:10.127035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8104a4-020e-4380-a467-f3ca85d8e17f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:10.237691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8104a4-020e-4380-a467-f3ca85d8e17f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:10.262056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:10.273373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815acd9-8832-430b-b2c3-31f8d20b3109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:10.274048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815acd9-8832-430b-b2c3-31f8d20b3109 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:10.274322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815acd9-8832-430b-b2c3-31f8d20b3109 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:10.274596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815acd9-8832-430b-b2c3-31f8d20b3109 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:10.275392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815acd9-8832-430b-b2c3-31f8d20b3109 None admin] RBAC: Adding query filter params (name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:10.276291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815acd9-8832-430b-b2c3-31f8d20b3109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:10.277585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815acd9-8832-430b-b2c3-31f8d20b3109 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:10.285820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 605/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:40:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:10.291994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6711362-8447-40d7-89ff-073a33278787 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:10.342271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:10.405957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6711362-8447-40d7-89ff-073a33278787 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:10.440705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727100a0-3cd5-4dc8-be53-4cf59b2293d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:10.441518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727100a0-3cd5-4dc8-be53-4cf59b2293d2 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:10.441783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727100a0-3cd5-4dc8-be53-4cf59b2293d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:10.442060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727100a0-3cd5-4dc8-be53-4cf59b2293d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:10.453366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727100a0-3cd5-4dc8-be53-4cf59b2293d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:10.454824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727100a0-3cd5-4dc8-be53-4cf59b2293d2 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-727100a0-3cd5-4dc8-be53-4cf59b2293d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:10.472351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:10.473439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:10.474571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:10.474571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:10.474571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:10.474571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:10.474571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 619/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:10.480068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815acd9-8832-430b-b2c3-31f8d20b3109 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:10.584151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815acd9-8832-430b-b2c3-31f8d20b3109 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:10.620065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8bc86-9028-4ebf-95db-69da54918d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:10.620725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8bc86-9028-4ebf-95db-69da54918d5d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:10.621012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8bc86-9028-4ebf-95db-69da54918d5d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:10.621294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8bc86-9028-4ebf-95db-69da54918d5d None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:10.622015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8bc86-9028-4ebf-95db-69da54918d5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:10.622327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8bc86-9028-4ebf-95db-69da54918d5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:10.623737 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8bc86-9028-4ebf-95db-69da54918d5d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:10.635588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 606/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:10.641548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727100a0-3cd5-4dc8-be53-4cf59b2293d2 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:10.736712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:10.743792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727100a0-3cd5-4dc8-be53-4cf59b2293d2 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:10.778151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de391c2-315b-4554-b10a-8f4f10e7a446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:10.778983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de391c2-315b-4554-b10a-8f4f10e7a446 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:10.779267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de391c2-315b-4554-b10a-8f4f10e7a446 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:10.779543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de391c2-315b-4554-b10a-8f4f10e7a446 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:10.787492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de391c2-315b-4554-b10a-8f4f10e7a446 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:10.788954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de391c2-315b-4554-b10a-8f4f10e7a446 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-8de391c2-315b-4554-b10a-8f4f10e7a446 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:40:10.806333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:10.808153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:10.809273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 620/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:10.814401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8bc86-9028-4ebf-95db-69da54918d5d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:10.867282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:10.925716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8bc86-9028-4ebf-95db-69da54918d5d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:10.965036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08ea60-1c5c-44cc-9cee-0a0af7ec9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:10.965719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08ea60-1c5c-44cc-9cee-0a0af7ec9a81 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:10.966008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08ea60-1c5c-44cc-9cee-0a0af7ec9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:10.966286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08ea60-1c5c-44cc-9cee-0a0af7ec9a81 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:10.967170 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08ea60-1c5c-44cc-9cee-0a0af7ec9a81 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:10.967486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08ea60-1c5c-44cc-9cee-0a0af7ec9a81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:10.968912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08ea60-1c5c-44cc-9cee-0a0af7ec9a81 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:10.987225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 607/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Jun 15 15:40:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:40:10.993779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de391c2-315b-4554-b10a-8f4f10e7a446 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:11.106223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de391c2-315b-4554-b10a-8f4f10e7a446 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:11.142865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6083ab5-2dec-4515-be27-e1e5312d79d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:11.143668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6083ab5-2dec-4515-be27-e1e5312d79d5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:11.143993 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6083ab5-2dec-4515-be27-e1e5312d79d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:11.144266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6083ab5-2dec-4515-be27-e1e5312d79d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:11.156126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6083ab5-2dec-4515-be27-e1e5312d79d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:11.157631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6083ab5-2dec-4515-be27-e1e5312d79d5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-f6083ab5-2dec-4515-be27-e1e5312d79d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:11.174724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:11.175820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:11.176766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:11.176766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:11.176766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:11.176766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:11.176766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 621/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:11.181687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08ea60-1c5c-44cc-9cee-0a0af7ec9a81 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:11.292836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08ea60-1c5c-44cc-9cee-0a0af7ec9a81 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:11.295849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:11.338507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca174ea-f4b2-4156-b1ad-114632ee9ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:11.339563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca174ea-f4b2-4156-b1ad-114632ee9ef4 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:11.339755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca174ea-f4b2-4156-b1ad-114632ee9ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:11.340110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca174ea-f4b2-4156-b1ad-114632ee9ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:11.341120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca174ea-f4b2-4156-b1ad-114632ee9ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:11.341426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca174ea-f4b2-4156-b1ad-114632ee9ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:11.343092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca174ea-f4b2-4156-b1ad-114632ee9ef4 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:11.359799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 608/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:11.366352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6083ab5-2dec-4515-be27-e1e5312d79d5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:11.379333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:11.476078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6083ab5-2dec-4515-be27-e1e5312d79d5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:11.517472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840ac41-95d1-4966-8678-d09e1748437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:11.518445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ac41-95d1-4966-8678-d09e1748437c None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:11.518838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ac41-95d1-4966-8678-d09e1748437c None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:11.519182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ac41-95d1-4966-8678-d09e1748437c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:11.527218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840ac41-95d1-4966-8678-d09e1748437c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:11.528778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840ac41-95d1-4966-8678-d09e1748437c None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-e840ac41-95d1-4966-8678-d09e1748437c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:40:11.544189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:11.545189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:11.546108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:40:11.546108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:40:11.546108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:40:11.546108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:40:11.546108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:40:11.546108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:40:11.546108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:11.546108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:11.546108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 622/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:40:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:11.552224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca174ea-f4b2-4156-b1ad-114632ee9ef4 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:11.667599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca174ea-f4b2-4156-b1ad-114632ee9ef4 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:11.702342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f2a58-6093-4df5-8a24-b7c1bcc4754f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:11.703034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f2a58-6093-4df5-8a24-b7c1bcc4754f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:11.703318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f2a58-6093-4df5-8a24-b7c1bcc4754f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:11.703597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f2a58-6093-4df5-8a24-b7c1bcc4754f None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:11.704378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f2a58-6093-4df5-8a24-b7c1bcc4754f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:11.704765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f2a58-6093-4df5-8a24-b7c1bcc4754f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:11.706103 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f2a58-6093-4df5-8a24-b7c1bcc4754f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:11.719208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 609/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:40:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:11.725589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840ac41-95d1-4966-8678-d09e1748437c None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:11.799071 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:11.839962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840ac41-95d1-4966-8678-d09e1748437c None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:11.877906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf3fc2-ee09-451b-8aa9-88bc23249690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:11.878927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf3fc2-ee09-451b-8aa9-88bc23249690 None admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:11.879248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf3fc2-ee09-451b-8aa9-88bc23249690 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:11.879521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf3fc2-ee09-451b-8aa9-88bc23249690 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/a786a95c48c348dbba269e0e33265271/roles/01737d3768e2407780ff51b345e1d020` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:11.895741 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf3fc2-ee09-451b-8aa9-88bc23249690 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=a786a95c48c348dbba269e0e33265271, role_id=01737d3768e2407780ff51b345e1d020)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:11.897343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf3fc2-ee09-451b-8aa9-88bc23249690 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:11.912318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 623/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:40:11 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/a786a95c48c348dbba269e0e33265271/roles/01737d3768e2407780ff51b345e1d020 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:40:14.215611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7493-c426-40bd-88f5-cb4688d04a23 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:14.216167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7493-c426-40bd-88f5-cb4688d04a23 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:14.216167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7493-c426-40bd-88f5-cb4688d04a23 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:14.217543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 610/1233] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:14.223539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b89425-2daf-4b45-a52d-1fb317ced122 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:14.223977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b89425-2daf-4b45-a52d-1fb317ced122 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:14.224097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b89425-2daf-4b45-a52d-1fb317ced122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:14.241701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-17b89425-2daf-4b45-a52d-1fb317ced122 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:14.243136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-17b89425-2daf-4b45-a52d-1fb317ced122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:14.325889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 624/1234] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:14.350633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a7493-c426-40bd-88f5-cb4688d04a23 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:14.378233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:14.481567 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a7493-c426-40bd-88f5-cb4688d04a23 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:14.521385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdd8db-0ee4-4309-8cbe-000b937052e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:14.522244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdd8db-0ee4-4309-8cbe-000b937052e7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:14.522529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdd8db-0ee4-4309-8cbe-000b937052e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:14.522844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdd8db-0ee4-4309-8cbe-000b937052e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:14.534947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdd8db-0ee4-4309-8cbe-000b937052e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-45cdd8db-0ee4-4309-8cbe-000b937052e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:40:14.543218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:14.544490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:14.545733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 611/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Jun 15 15:40:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:14.550678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b89425-2daf-4b45-a52d-1fb317ced122 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:14.602431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:14.666025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b89425-2daf-4b45-a52d-1fb317ced122 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:14.705671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e75ac4-ac33-4bbe-a07f-3e5ac89033ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:14.706340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e75ac4-ac33-4bbe-a07f-3e5ac89033ef None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:14.706620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e75ac4-ac33-4bbe-a07f-3e5ac89033ef None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:14.706921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e75ac4-ac33-4bbe-a07f-3e5ac89033ef None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:14.707658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e75ac4-ac33-4bbe-a07f-3e5ac89033ef None admin] RBAC: Adding query filter params (name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:14.707963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e75ac4-ac33-4bbe-a07f-3e5ac89033ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:14.709331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e75ac4-ac33-4bbe-a07f-3e5ac89033ef None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:14.719376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 625/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Jun 15 15:40:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:14.725247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdd8db-0ee4-4309-8cbe-000b937052e7 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:14.819939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:14.839314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdd8db-0ee4-4309-8cbe-000b937052e7 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:14.876247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f9990-da3a-4585-8171-1db729c78f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:14.877232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f9990-da3a-4585-8171-1db729c78f21 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:14.877582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f9990-da3a-4585-8171-1db729c78f21 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:14.877905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f9990-da3a-4585-8171-1db729c78f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:14.891579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f9990-da3a-4585-8171-1db729c78f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:14.893145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f9990-da3a-4585-8171-1db729c78f21 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-fc1f9990-da3a-4585-8171-1db729c78f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:14.912632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:14.913768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:14.915014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:14.915014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:14.915014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:14.915014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:14.915014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 612/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:14.919897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e75ac4-ac33-4bbe-a07f-3e5ac89033ef None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:15.032313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e75ac4-ac33-4bbe-a07f-3e5ac89033ef None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:15.043474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:15.073187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0be104-736f-4b22-8cf2-6f75b5765933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:15.073873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0be104-736f-4b22-8cf2-6f75b5765933 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:15.074163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0be104-736f-4b22-8cf2-6f75b5765933 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:15.074442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0be104-736f-4b22-8cf2-6f75b5765933 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:15.075200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0be104-736f-4b22-8cf2-6f75b5765933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:15.075510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0be104-736f-4b22-8cf2-6f75b5765933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:15.076896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0be104-736f-4b22-8cf2-6f75b5765933 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:15.089368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 626/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:15.095642 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f9990-da3a-4585-8171-1db729c78f21 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:15.211794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f9990-da3a-4585-8171-1db729c78f21 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:15.253884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbdb39-940d-49e5-bd07-299cab90a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:15.254761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbdb39-940d-49e5-bd07-299cab90a80a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:15.255028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbdb39-940d-49e5-bd07-299cab90a80a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:15.255358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbdb39-940d-49e5-bd07-299cab90a80a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:15.259085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:15.268196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbdb39-940d-49e5-bd07-299cab90a80a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:15.269851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbdb39-940d-49e5-bd07-299cab90a80a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-35cbdb39-940d-49e5-bd07-299cab90a80a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:40:15.286410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:15.287429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:15.289053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 613/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:15.294584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0be104-736f-4b22-8cf2-6f75b5765933 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:15.412492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0be104-736f-4b22-8cf2-6f75b5765933 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:15.453286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e474f90-ec1b-40b5-ac5c-4ec80bdc3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:15.453963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e474f90-ec1b-40b5-ac5c-4ec80bdc3e27 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:15.454275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e474f90-ec1b-40b5-ac5c-4ec80bdc3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:15.454553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e474f90-ec1b-40b5-ac5c-4ec80bdc3e27 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:15.455422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e474f90-ec1b-40b5-ac5c-4ec80bdc3e27 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:15.455732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e474f90-ec1b-40b5-ac5c-4ec80bdc3e27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:15.457167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e474f90-ec1b-40b5-ac5c-4ec80bdc3e27 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:15.477001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 627/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Jun 15 15:40:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:15.483430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbdb39-940d-49e5-bd07-299cab90a80a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:15.594936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbdb39-940d-49e5-bd07-299cab90a80a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:15.632369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe28ab-ad6c-403c-bc92-8d684a7e914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:15.633043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe28ab-ad6c-403c-bc92-8d684a7e914a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:15.633326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe28ab-ad6c-403c-bc92-8d684a7e914a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:15.633613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe28ab-ad6c-403c-bc92-8d684a7e914a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:15.704931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:15.712268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe28ab-ad6c-403c-bc92-8d684a7e914a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:15.714353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe28ab-ad6c-403c-bc92-8d684a7e914a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:15.826782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 614/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:40:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:40:15.833223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e474f90-ec1b-40b5-ac5c-4ec80bdc3e27 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:15.864795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:15.948947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e474f90-ec1b-40b5-ac5c-4ec80bdc3e27 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:15.985581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b172f7-914e-4a24-9fdc-5ca7ebd2850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:15.986348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b172f7-914e-4a24-9fdc-5ca7ebd2850b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:15.986630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b172f7-914e-4a24-9fdc-5ca7ebd2850b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:15.986997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b172f7-914e-4a24-9fdc-5ca7ebd2850b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:16.000832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b172f7-914e-4a24-9fdc-5ca7ebd2850b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:16.002961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b172f7-914e-4a24-9fdc-5ca7ebd2850b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-c4b172f7-914e-4a24-9fdc-5ca7ebd2850b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:16.034087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:16.035287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:16.036450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:16.036450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:16.036450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:16.036450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:16.036450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 628/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:16.046325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe28ab-ad6c-403c-bc92-8d684a7e914a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:16.183835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:16.190599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe28ab-ad6c-403c-bc92-8d684a7e914a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:16.232922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2545f43-3ff5-42a8-9b0c-41d9ea7091ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:16.233657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2545f43-3ff5-42a8-9b0c-41d9ea7091ac None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:16.233935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2545f43-3ff5-42a8-9b0c-41d9ea7091ac None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:16.234356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2545f43-3ff5-42a8-9b0c-41d9ea7091ac None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:16.235158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2545f43-3ff5-42a8-9b0c-41d9ea7091ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:16.235667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2545f43-3ff5-42a8-9b0c-41d9ea7091ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:16.237512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2545f43-3ff5-42a8-9b0c-41d9ea7091ac None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:16.258355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 615/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:16.259972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b172f7-914e-4a24-9fdc-5ca7ebd2850b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:16.345517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:16.369741 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b172f7-914e-4a24-9fdc-5ca7ebd2850b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:16.407886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e351ea-1463-415e-984e-06bff1dacf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:16.408683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e351ea-1463-415e-984e-06bff1dacf20 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:16.408962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e351ea-1463-415e-984e-06bff1dacf20 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:16.409262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e351ea-1463-415e-984e-06bff1dacf20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:16.416534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e351ea-1463-415e-984e-06bff1dacf20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:16.418048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e351ea-1463-415e-984e-06bff1dacf20 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-38e351ea-1463-415e-984e-06bff1dacf20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:40:16.434716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:16.435712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:16.436681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:40:16.436681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:40:16.436681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:40:16.436681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:40:16.436681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:40:16.436681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:40:16.436681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:16.436681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:16.440937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 629/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:40:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:16.442300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2545f43-3ff5-42a8-9b0c-41d9ea7091ac None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:16.565193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2545f43-3ff5-42a8-9b0c-41d9ea7091ac None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:16.602022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c459f-9a1d-444a-8633-f314ab98c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:16.602695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c459f-9a1d-444a-8633-f314ab98c987 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:16.603010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c459f-9a1d-444a-8633-f314ab98c987 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:16.603326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c459f-9a1d-444a-8633-f314ab98c987 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:16.604125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c459f-9a1d-444a-8633-f314ab98c987 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:16.604441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c459f-9a1d-444a-8633-f314ab98c987 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:16.605888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c459f-9a1d-444a-8633-f314ab98c987 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:16.618577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 616/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:40:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:16.625710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e351ea-1463-415e-984e-06bff1dacf20 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:16.730691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e351ea-1463-415e-984e-06bff1dacf20 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:16.767813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bf16b-7a93-488d-9efe-4c7a6816cf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:16.768485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bf16b-7a93-488d-9efe-4c7a6816cf39 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:16.768878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bf16b-7a93-488d-9efe-4c7a6816cf39 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:16.769202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bf16b-7a93-488d-9efe-4c7a6816cf39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:16.770221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bf16b-7a93-488d-9efe-4c7a6816cf39 None admin] RBAC: Adding query filter params (role.id=01737d3768e2407780ff51b345e1d020, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=a786a95c48c348dbba269e0e33265271) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:16.770622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bf16b-7a93-488d-9efe-4c7a6816cf39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:16.772186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bf16b-7a93-488d-9efe-4c7a6816cf39 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:16.780116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 630/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:40:16 2022] GET /identity/v3/role_assignments?user.id=a786a95c48c348dbba269e0e33265271&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=01737d3768e2407780ff51b345e1d020 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:40:19.121951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d571-3d27-4f4c-bb6b-a014790aeefc None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:19.122613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d571-3d27-4f4c-bb6b-a014790aeefc None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:19.122613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d571-3d27-4f4c-bb6b-a014790aeefc None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:19.124038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 617/1247] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:19.129518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c71b45-43a9-4372-b10f-b6c0dcd5a1b0 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:19.129957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c71b45-43a9-4372-b10f-b6c0dcd5a1b0 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:19.130085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c71b45-43a9-4372-b10f-b6c0dcd5a1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:19.136140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:19.150813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-67c71b45-43a9-4372-b10f-b6c0dcd5a1b0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:19.152030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-67c71b45-43a9-4372-b10f-b6c0dcd5a1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:19.232824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 631/1248] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:19.260727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144d571-3d27-4f4c-bb6b-a014790aeefc None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:19.284355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:19.386938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144d571-3d27-4f4c-bb6b-a014790aeefc None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:19.427248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c1c71-315f-4130-81fa-21e306e3ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:19.428037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c1c71-315f-4130-81fa-21e306e3ace1 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:19.428329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c1c71-315f-4130-81fa-21e306e3ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:19.428620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c1c71-315f-4130-81fa-21e306e3ace1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:19.441426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c1c71-315f-4130-81fa-21e306e3ace1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-e59c1c71-315f-4130-81fa-21e306e3ace1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:40:19.449844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:19.450885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:40:19.452246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:19.456124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 618/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:40:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:19.457266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c71b45-43a9-4372-b10f-b6c0dcd5a1b0 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:19.569799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c71b45-43a9-4372-b10f-b6c0dcd5a1b0 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:19.602228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:19.608648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f4e02-f28f-47c1-b3e4-b503b2a80353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:19.609321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f4e02-f28f-47c1-b3e4-b503b2a80353 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:19.609619 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f4e02-f28f-47c1-b3e4-b503b2a80353 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:19.609882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f4e02-f28f-47c1-b3e4-b503b2a80353 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:19.610636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f4e02-f28f-47c1-b3e4-b503b2a80353 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:19.611013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f4e02-f28f-47c1-b3e4-b503b2a80353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:19.612385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f4e02-f28f-47c1-b3e4-b503b2a80353 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:19.622004 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 632/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:40:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:19.627954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c1c71-315f-4130-81fa-21e306e3ace1 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:19.710565 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:19.734361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c1c71-315f-4130-81fa-21e306e3ace1 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:19.769007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942618e-886d-4b34-9efe-b30d4d9ba41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:19.769798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942618e-886d-4b34-9efe-b30d4d9ba41d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:19.770083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942618e-886d-4b34-9efe-b30d4d9ba41d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:19.770362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942618e-886d-4b34-9efe-b30d4d9ba41d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:19.782856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942618e-886d-4b34-9efe-b30d4d9ba41d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:19.784516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942618e-886d-4b34-9efe-b30d4d9ba41d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-2942618e-886d-4b34-9efe-b30d4d9ba41d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:19.804428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:19.805428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:19.806939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:19.806939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:19.806939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:19.806939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:19.812329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 619/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:19.813442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f4e02-f28f-47c1-b3e4-b503b2a80353 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:19.939107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f4e02-f28f-47c1-b3e4-b503b2a80353 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:19.977186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3107c-58fd-4d31-9f62-a43e3624c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:19.977816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3107c-58fd-4d31-9f62-a43e3624c009 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:19.978121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3107c-58fd-4d31-9f62-a43e3624c009 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:19.978356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3107c-58fd-4d31-9f62-a43e3624c009 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:19.979155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3107c-58fd-4d31-9f62-a43e3624c009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:19.979464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3107c-58fd-4d31-9f62-a43e3624c009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:19.980900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3107c-58fd-4d31-9f62-a43e3624c009 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:19.992945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 633/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:19.999325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942618e-886d-4b34-9efe-b30d4d9ba41d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:20.118380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942618e-886d-4b34-9efe-b30d4d9ba41d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:20.146365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:20.155479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83984446-93df-4af4-ade5-8a7fd7dd7cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:20.156572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83984446-93df-4af4-ade5-8a7fd7dd7cce None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:20.156965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83984446-93df-4af4-ade5-8a7fd7dd7cce None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:20.157333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83984446-93df-4af4-ade5-8a7fd7dd7cce None admin] PATH_INFO: `/v3/users/placement` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:20.165164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83984446-93df-4af4-ade5-8a7fd7dd7cce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:20.167199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83984446-93df-4af4-ade5-8a7fd7dd7cce None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-83984446-93df-4af4-ade5-8a7fd7dd7cce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:40:20.181937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:20.183015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:20.184301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 620/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:20.189262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3107c-58fd-4d31-9f62-a43e3624c009 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:20.258847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:20.308761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3107c-58fd-4d31-9f62-a43e3624c009 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:20.344816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9bcfa-a2d6-447c-a8ee-b78e78859ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:20.345531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9bcfa-a2d6-447c-a8ee-b78e78859ab0 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:20.345807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9bcfa-a2d6-447c-a8ee-b78e78859ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:20.346084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9bcfa-a2d6-447c-a8ee-b78e78859ab0 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:20.346944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9bcfa-a2d6-447c-a8ee-b78e78859ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:20.347254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9bcfa-a2d6-447c-a8ee-b78e78859ab0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:20.348688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9bcfa-a2d6-447c-a8ee-b78e78859ab0 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:20.366934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 634/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Jun 15 15:40:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:40:20.373233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83984446-93df-4af4-ade5-8a7fd7dd7cce None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:20.478476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83984446-93df-4af4-ade5-8a7fd7dd7cce None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:20.516024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afc81c-ebe3-4cfc-9366-b42c449f408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:20.516916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afc81c-ebe3-4cfc-9366-b42c449f408c None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:20.517210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afc81c-ebe3-4cfc-9366-b42c449f408c None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:20.517536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afc81c-ebe3-4cfc-9366-b42c449f408c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:20.578945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:20.592346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afc81c-ebe3-4cfc-9366-b42c449f408c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:20.594010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afc81c-ebe3-4cfc-9366-b42c449f408c None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:20.699212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 621/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:40:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:40:20.705267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9bcfa-a2d6-447c-a8ee-b78e78859ab0 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:20.814678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9bcfa-a2d6-447c-a8ee-b78e78859ab0 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:20.829977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:20.853904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca1846-8955-4803-9505-9db5c881eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:20.854874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca1846-8955-4803-9505-9db5c881eea9 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:20.855179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca1846-8955-4803-9505-9db5c881eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:20.855522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca1846-8955-4803-9505-9db5c881eea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:20.866499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca1846-8955-4803-9505-9db5c881eea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:20.868227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca1846-8955-4803-9505-9db5c881eea9 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-90ca1846-8955-4803-9505-9db5c881eea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:20.886915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:20.888416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:20.889527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:20.889527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:20.889527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:20.889527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:20.889527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 635/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:20.894525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afc81c-ebe3-4cfc-9366-b42c449f408c None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:20.994362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:21.007395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afc81c-ebe3-4cfc-9366-b42c449f408c None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:21.043916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758c034-1a0c-4201-a101-061f083fbf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:21.044563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758c034-1a0c-4201-a101-061f083fbf13 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:21.044863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758c034-1a0c-4201-a101-061f083fbf13 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:21.045211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758c034-1a0c-4201-a101-061f083fbf13 None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:21.045996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758c034-1a0c-4201-a101-061f083fbf13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:21.046306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758c034-1a0c-4201-a101-061f083fbf13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:21.047789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758c034-1a0c-4201-a101-061f083fbf13 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:21.060239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 622/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:21.066808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca1846-8955-4803-9505-9db5c881eea9 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:21.178304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca1846-8955-4803-9505-9db5c881eea9 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:21.217348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bb562-da16-4b90-939f-bd499643160e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:21.218407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bb562-da16-4b90-939f-bd499643160e None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:21.218691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bb562-da16-4b90-939f-bd499643160e None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:21.219070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bb562-da16-4b90-939f-bd499643160e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:21.227369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bb562-da16-4b90-939f-bd499643160e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:21.229763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bb562-da16-4b90-939f-bd499643160e None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-c58bb562-da16-4b90-939f-bd499643160e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:40:21.245483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:21.246519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:21.247837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:40:21.247837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:40:21.247837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:40:21.247837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:40:21.247837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:40:21.247837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:40:21.247837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:21.247837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:21.247837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 636/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:40:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:21.253478 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758c034-1a0c-4201-a101-061f083fbf13 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:21.365356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758c034-1a0c-4201-a101-061f083fbf13 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:21.401570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc0f00-c9cb-48eb-8e72-226bef69c937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:21.402236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0f00-c9cb-48eb-8e72-226bef69c937 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:21.402520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0f00-c9cb-48eb-8e72-226bef69c937 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:21.402829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0f00-c9cb-48eb-8e72-226bef69c937 None admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:21.403625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc0f00-c9cb-48eb-8e72-226bef69c937 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:21.403937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc0f00-c9cb-48eb-8e72-226bef69c937 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:21.405393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc0f00-c9cb-48eb-8e72-226bef69c937 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:21.419122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 623/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:40:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:21.425312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bb562-da16-4b90-939f-bd499643160e None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:21.457981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:21.536223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bb562-da16-4b90-939f-bd499643160e None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:21.572604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c5276-dd03-484c-bbca-5b867a9fa5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:21.573294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c5276-dd03-484c-bbca-5b867a9fa5f3 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:21.573559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c5276-dd03-484c-bbca-5b867a9fa5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:21.573835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c5276-dd03-484c-bbca-5b867a9fa5f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:21.574676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c5276-dd03-484c-bbca-5b867a9fa5f3 None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=a786a95c48c348dbba269e0e33265271) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:21.575045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c5276-dd03-484c-bbca-5b867a9fa5f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:21.576590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c5276-dd03-484c-bbca-5b867a9fa5f3 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:21.582924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 637/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:40:21 2022] GET /identity/v3/role_assignments?user.id=a786a95c48c348dbba269e0e33265271&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=433478cc318a467c931f13a757f161fa => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:23.687599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6d471-8026-48bd-b470-dc5847fc0121 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:23.688118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6d471-8026-48bd-b470-dc5847fc0121 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:23.688238 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6d471-8026-48bd-b470-dc5847fc0121 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:23.689495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 624/1261] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:23.694591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712bded-d457-4415-91b9-e1c15449d375 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:23.695022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712bded-d457-4415-91b9-e1c15449d375 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:23.695199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712bded-d457-4415-91b9-e1c15449d375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:23.712422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-0712bded-d457-4415-91b9-e1c15449d375 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:23.713543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0712bded-d457-4415-91b9-e1c15449d375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:23.778190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 638/1262] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:23.800419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6d471-8026-48bd-b470-dc5847fc0121 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:23.811639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:23.899596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6d471-8026-48bd-b470-dc5847fc0121 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:23.933042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624658bf-446b-4d4a-90ae-b2c767e62d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:23.933818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624658bf-446b-4d4a-90ae-b2c767e62d6e None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:23.934105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624658bf-446b-4d4a-90ae-b2c767e62d6e None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:23.934385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624658bf-446b-4d4a-90ae-b2c767e62d6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:23.944172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624658bf-446b-4d4a-90ae-b2c767e62d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-624658bf-446b-4d4a-90ae-b2c767e62d6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:40:23.951775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:23.952686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:23.953736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 625/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:40:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:23.959044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712bded-d457-4415-91b9-e1c15449d375 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:23.980612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:24.058831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712bded-d457-4415-91b9-e1c15449d375 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:24.090877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf9a5-b353-4bd9-8ce0-747234597a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:24.091525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf9a5-b353-4bd9-8ce0-747234597a55 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:24.091804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf9a5-b353-4bd9-8ce0-747234597a55 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:24.092083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf9a5-b353-4bd9-8ce0-747234597a55 None admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:24.092806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf9a5-b353-4bd9-8ce0-747234597a55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:24.093107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf9a5-b353-4bd9-8ce0-747234597a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:24.094456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf9a5-b353-4bd9-8ce0-747234597a55 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:24.102152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 639/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:40:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:24.108423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624658bf-446b-4d4a-90ae-b2c767e62d6e None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:24.173589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:24.206194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624658bf-446b-4d4a-90ae-b2c767e62d6e None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:24.244862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686e604-7d57-472e-a7d5-9435e75e983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:24.245664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686e604-7d57-472e-a7d5-9435e75e983d None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:24.245951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686e604-7d57-472e-a7d5-9435e75e983d None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:24.246282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686e604-7d57-472e-a7d5-9435e75e983d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:24.257689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686e604-7d57-472e-a7d5-9435e75e983d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:24.259661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686e604-7d57-472e-a7d5-9435e75e983d None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-c686e604-7d57-472e-a7d5-9435e75e983d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:24.281981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:24.283106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:24.284405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:24.284405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:24.284405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:24.284405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:24.284405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 626/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:24.288981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf9a5-b353-4bd9-8ce0-747234597a55 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:24.367627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:24.386904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf9a5-b353-4bd9-8ce0-747234597a55 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:24.420406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85793822-0576-4323-bc83-41d4ee2454e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:24.421058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85793822-0576-4323-bc83-41d4ee2454e5 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:24.421340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85793822-0576-4323-bc83-41d4ee2454e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:24.421616 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85793822-0576-4323-bc83-41d4ee2454e5 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:24.422323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85793822-0576-4323-bc83-41d4ee2454e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:24.422628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85793822-0576-4323-bc83-41d4ee2454e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:24.424076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85793822-0576-4323-bc83-41d4ee2454e5 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:24.435607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 640/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:24.441237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686e604-7d57-472e-a7d5-9435e75e983d None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:24.537852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686e604-7d57-472e-a7d5-9435e75e983d None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:24.556311 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:24.570156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49585bb8-8eef-4f80-87d3-908688b98e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:24.570959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49585bb8-8eef-4f80-87d3-908688b98e0f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:24.571240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49585bb8-8eef-4f80-87d3-908688b98e0f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:24.571508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49585bb8-8eef-4f80-87d3-908688b98e0f None admin] PATH_INFO: `/v3/users/placement` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:24.579905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49585bb8-8eef-4f80-87d3-908688b98e0f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:24.581318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49585bb8-8eef-4f80-87d3-908688b98e0f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-49585bb8-8eef-4f80-87d3-908688b98e0f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:40:24.595657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:24.596768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:24.597981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 627/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:24.602483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85793822-0576-4323-bc83-41d4ee2454e5 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:24.699100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85793822-0576-4323-bc83-41d4ee2454e5 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:24.736165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014cce-340b-498d-bbc8-29ea19598161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:24.736818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014cce-340b-498d-bbc8-29ea19598161 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:24.737104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014cce-340b-498d-bbc8-29ea19598161 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:24.737384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014cce-340b-498d-bbc8-29ea19598161 None admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:24.738210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19014cce-340b-498d-bbc8-29ea19598161 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:24.738523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19014cce-340b-498d-bbc8-29ea19598161 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:24.739999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19014cce-340b-498d-bbc8-29ea19598161 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:24.749139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:24.757431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 641/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Jun 15 15:40:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:24.763911 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49585bb8-8eef-4f80-87d3-908688b98e0f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:24.858639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49585bb8-8eef-4f80-87d3-908688b98e0f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:24.891226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5800a03-6537-4f0a-81ad-c468fd6c2017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:24.891989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5800a03-6537-4f0a-81ad-c468fd6c2017 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:24.892267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5800a03-6537-4f0a-81ad-c468fd6c2017 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:24.892541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5800a03-6537-4f0a-81ad-c468fd6c2017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:24.902717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5800a03-6537-4f0a-81ad-c468fd6c2017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:24.904159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5800a03-6537-4f0a-81ad-c468fd6c2017 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-a5800a03-6537-4f0a-81ad-c468fd6c2017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:24.920597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:24.921662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:24.922800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:24.922800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:24.922800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:24.922800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:24.926211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 628/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:24.927386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014cce-340b-498d-bbc8-29ea19598161 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:25.023551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014cce-340b-498d-bbc8-29ea19598161 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:25.062083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09d044-acaf-4cd6-a6cf-6445cf298801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:25.062755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09d044-acaf-4cd6-a6cf-6445cf298801 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:25.063049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09d044-acaf-4cd6-a6cf-6445cf298801 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:25.063328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09d044-acaf-4cd6-a6cf-6445cf298801 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:25.064037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09d044-acaf-4cd6-a6cf-6445cf298801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:25.064342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09d044-acaf-4cd6-a6cf-6445cf298801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:25.065700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09d044-acaf-4cd6-a6cf-6445cf298801 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:25.076533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 642/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:25.082164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5800a03-6537-4f0a-81ad-c468fd6c2017 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:25.113383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:25.180407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5800a03-6537-4f0a-81ad-c468fd6c2017 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:25.216055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bd2fe-59c0-486a-b8b2-673a9b5c3659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:25.217081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bd2fe-59c0-486a-b8b2-673a9b5c3659 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:25.217435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bd2fe-59c0-486a-b8b2-673a9b5c3659 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:25.217765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bd2fe-59c0-486a-b8b2-673a9b5c3659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:25.226648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bd2fe-59c0-486a-b8b2-673a9b5c3659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:25.229345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bd2fe-59c0-486a-b8b2-673a9b5c3659 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-cc0bd2fe-59c0-486a-b8b2-673a9b5c3659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:40:25.249571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:25.250477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:25.251650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:40:25.251650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:40:25.251650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:40:25.251650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:40:25.251650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:40:25.251650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:40:25.251650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:25.251650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:25.251650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 629/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:40:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:25.257530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09d044-acaf-4cd6-a6cf-6445cf298801 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:25.315303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:25.355535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09d044-acaf-4cd6-a6cf-6445cf298801 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:25.388013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d02c4a-aa09-46e1-94d0-8a6539ab3a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:25.388660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d02c4a-aa09-46e1-94d0-8a6539ab3a8d None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:25.388941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d02c4a-aa09-46e1-94d0-8a6539ab3a8d None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:25.389213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d02c4a-aa09-46e1-94d0-8a6539ab3a8d None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:25.389987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d02c4a-aa09-46e1-94d0-8a6539ab3a8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:25.390296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d02c4a-aa09-46e1-94d0-8a6539ab3a8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:25.391763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d02c4a-aa09-46e1-94d0-8a6539ab3a8d None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:25.406042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 643/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:25.411975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bd2fe-59c0-486a-b8b2-673a9b5c3659 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:25.501741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:25.510846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bd2fe-59c0-486a-b8b2-673a9b5c3659 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:25.544516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee6a8a-ae68-4a78-9fd4-a70c1425bcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:25.545392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee6a8a-ae68-4a78-9fd4-a70c1425bcf2 None admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:25.545676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee6a8a-ae68-4a78-9fd4-a70c1425bcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:25.545957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee6a8a-ae68-4a78-9fd4-a70c1425bcf2 None admin] PATH_INFO: `/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/a786a95c48c348dbba269e0e33265271/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:25.547916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee6a8a-ae68-4a78-9fd4-a70c1425bcf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aa8cd5804554314b5b0a61cb3c8ec9d, user_id=a786a95c48c348dbba269e0e33265271, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:25.549333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee6a8a-ae68-4a78-9fd4-a70c1425bcf2 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:25.561972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 630/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Jun 15 15:40:25 2022] PUT /identity/v3/projects/9aa8cd5804554314b5b0a61cb3c8ec9d/users/a786a95c48c348dbba269e0e33265271/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:40:27.644952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4565f38-9534-4db3-beb2-21346bcb4c3f None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:27.645544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4565f38-9534-4db3-beb2-21346bcb4c3f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:27.645544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4565f38-9534-4db3-beb2-21346bcb4c3f None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:27.646888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 644/1274] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:27.652039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec473-886f-445b-bfdf-394babe2dfee None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:27.652475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec473-886f-445b-bfdf-394babe2dfee None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:27.652584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec473-886f-445b-bfdf-394babe2dfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:27.669184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-82eec473-886f-445b-bfdf-394babe2dfee None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:27.670343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-82eec473-886f-445b-bfdf-394babe2dfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:27.734391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 631/1275] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:27.757752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4565f38-9534-4db3-beb2-21346bcb4c3f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:27.856819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4565f38-9534-4db3-beb2-21346bcb4c3f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:27.873438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:27.890487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4f29b-d141-463f-975a-5426973216cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:27.891299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4f29b-d141-463f-975a-5426973216cf None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:27.891586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4f29b-d141-463f-975a-5426973216cf None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:27.891861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4f29b-d141-463f-975a-5426973216cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:27.902373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4f29b-d141-463f-975a-5426973216cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-38b4f29b-d141-463f-975a-5426973216cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 15:40:27.909992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:27.911199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:27.912143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 645/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Jun 15 15:40:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:27.917555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eec473-886f-445b-bfdf-394babe2dfee None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:28.003044 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:28.021462 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eec473-886f-445b-bfdf-394babe2dfee None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:28.058494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7430ed-8627-4178-af2d-c12adc4e4412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:28.059187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7430ed-8627-4178-af2d-c12adc4e4412 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:28.059476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7430ed-8627-4178-af2d-c12adc4e4412 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:28.059756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7430ed-8627-4178-af2d-c12adc4e4412 None admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:28.060503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7430ed-8627-4178-af2d-c12adc4e4412 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:28.060814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7430ed-8627-4178-af2d-c12adc4e4412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:28.062190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7430ed-8627-4178-af2d-c12adc4e4412 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:28.070061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 632/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Jun 15 15:40:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:28.076298 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4f29b-d141-463f-975a-5426973216cf None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:28.172392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4f29b-d141-463f-975a-5426973216cf None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:28.204948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98268979-7616-4fc9-a30a-2ff030e4f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:28.205850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98268979-7616-4fc9-a30a-2ff030e4f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:28.206169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98268979-7616-4fc9-a30a-2ff030e4f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:28.206451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98268979-7616-4fc9-a30a-2ff030e4f2d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:28.219010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98268979-7616-4fc9-a30a-2ff030e4f2d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:28.220788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98268979-7616-4fc9-a30a-2ff030e4f2d6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-98268979-7616-4fc9-a30a-2ff030e4f2d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:28.240768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:28.241874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:28.243015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:28.243015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:28.243015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:28.243015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:28.243015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 646/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:28.247914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7430ed-8627-4178-af2d-c12adc4e4412 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:28.346020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7430ed-8627-4178-af2d-c12adc4e4412 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:28.377850 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54628d-1d11-4991-bbde-d02840dbda1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:28.378502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54628d-1d11-4991-bbde-d02840dbda1a None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:28.378817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54628d-1d11-4991-bbde-d02840dbda1a None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:28.379101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54628d-1d11-4991-bbde-d02840dbda1a None admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:28.379810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54628d-1d11-4991-bbde-d02840dbda1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:28.380119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54628d-1d11-4991-bbde-d02840dbda1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:28.381468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54628d-1d11-4991-bbde-d02840dbda1a None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:28.393586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 633/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:28.398994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98268979-7616-4fc9-a30a-2ff030e4f2d6 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:28.429231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:28.498564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98268979-7616-4fc9-a30a-2ff030e4f2d6 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:28.531090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cb97c-74a5-416f-8cb0-26e15dc217aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:28.531873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cb97c-74a5-416f-8cb0-26e15dc217aa None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:28.532155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cb97c-74a5-416f-8cb0-26e15dc217aa None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:28.532432 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cb97c-74a5-416f-8cb0-26e15dc217aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:28.539571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41cb97c-74a5-416f-8cb0-26e15dc217aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:28.541038 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41cb97c-74a5-416f-8cb0-26e15dc217aa None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-e41cb97c-74a5-416f-8cb0-26e15dc217aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 15:40:28.554955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:28.556607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:28.557931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 647/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:28.562273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54628d-1d11-4991-bbde-d02840dbda1a None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:28.565992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:28.658008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54628d-1d11-4991-bbde-d02840dbda1a None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:28.690533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b6f19-767d-45ff-bcd4-cb63ff6f70c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:28.691213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b6f19-767d-45ff-bcd4-cb63ff6f70c5 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:28.691497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b6f19-767d-45ff-bcd4-cb63ff6f70c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:28.691775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b6f19-767d-45ff-bcd4-cb63ff6f70c5 None admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:28.692599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b6f19-767d-45ff-bcd4-cb63ff6f70c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:28.692907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b6f19-767d-45ff-bcd4-cb63ff6f70c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:28.694306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b6f19-767d-45ff-bcd4-cb63ff6f70c5 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:28.711055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 634/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Jun 15 15:40:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:28.717501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cb97c-74a5-416f-8cb0-26e15dc217aa None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:28.808690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:28.818443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cb97c-74a5-416f-8cb0-26e15dc217aa None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:28.850252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17b43-dc85-46af-a7e2-a8aa20fc6228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:28.850914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17b43-dc85-46af-a7e2-a8aa20fc6228 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:28.851197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17b43-dc85-46af-a7e2-a8aa20fc6228 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:28.851472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17b43-dc85-46af-a7e2-a8aa20fc6228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:28.916613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d17b43-dc85-46af-a7e2-a8aa20fc6228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:28.917980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d17b43-dc85-46af-a7e2-a8aa20fc6228 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:29.002683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:29.013052 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 648/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Jun 15 15:40:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 15:40:29.018928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b6f19-767d-45ff-bcd4-cb63ff6f70c5 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:29.079785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:29.117510 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b6f19-767d-45ff-bcd4-cb63ff6f70c5 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:29.150380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde59392-69d2-47b7-8c90-1bd8817c239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:29.151185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde59392-69d2-47b7-8c90-1bd8817c239b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:29.151472 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde59392-69d2-47b7-8c90-1bd8817c239b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:29.151750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde59392-69d2-47b7-8c90-1bd8817c239b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:29.161331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde59392-69d2-47b7-8c90-1bd8817c239b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:29.162765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde59392-69d2-47b7-8c90-1bd8817c239b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-bde59392-69d2-47b7-8c90-1bd8817c239b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 15:40:29.180075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:29.181784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:29.182752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 15:40:29.182752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 15:40:29.182752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 15:40:29.182752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:29.182752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 635/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:29.187400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17b43-dc85-46af-a7e2-a8aa20fc6228 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:29.283740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17b43-dc85-46af-a7e2-a8aa20fc6228 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:29.316405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd04a-cefc-4053-87e5-f7713247f725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:29.317058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd04a-cefc-4053-87e5-f7713247f725 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:29.317342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd04a-cefc-4053-87e5-f7713247f725 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:29.317621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd04a-cefc-4053-87e5-f7713247f725 None admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:29.318342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cd04a-cefc-4053-87e5-f7713247f725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:29.318651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cd04a-cefc-4053-87e5-f7713247f725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:29.320043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cd04a-cefc-4053-87e5-f7713247f725 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:29.330921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 649/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Jun 15 15:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:29.336543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde59392-69d2-47b7-8c90-1bd8817c239b None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:29.433492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde59392-69d2-47b7-8c90-1bd8817c239b None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:29.447208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:29.467976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ce082-106a-4ab1-81a9-491cc67d306f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:29.468750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ce082-106a-4ab1-81a9-491cc67d306f None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:29.469032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ce082-106a-4ab1-81a9-491cc67d306f None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:29.469307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ce082-106a-4ab1-81a9-491cc67d306f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:29.475758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ce082-106a-4ab1-81a9-491cc67d306f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:29.477193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ce082-106a-4ab1-81a9-491cc67d306f None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application [None req-c85ce082-106a-4ab1-81a9-491cc67d306f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 15:40:29.491206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:29.492272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:29.493249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 15:40:29.493249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 15:40:29.493249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 15:40:29.493249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 15:40:29.493249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 15:40:29.493249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 15:40:29.493249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 15:40:29.493249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: ERROR keystone.server.flask.application Jun 15 15:40:29.493249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 636/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Jun 15 15:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:29.498096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd04a-cefc-4053-87e5-f7713247f725 None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:29.548126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:29.597014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd04a-cefc-4053-87e5-f7713247f725 None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:29.628555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a653d-cf4a-4ab9-a899-553bf66e1af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:29.629194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a653d-cf4a-4ab9-a899-553bf66e1af6 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:29.629473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a653d-cf4a-4ab9-a899-553bf66e1af6 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:29.629745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a653d-cf4a-4ab9-a899-553bf66e1af6 None admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:29.630506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a653d-cf4a-4ab9-a899-553bf66e1af6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:29.630837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a653d-cf4a-4ab9-a899-553bf66e1af6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:29.632257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a653d-cf4a-4ab9-a899-553bf66e1af6 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:29.644602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 650/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Jun 15 15:40:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:29.651154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ce082-106a-4ab1-81a9-491cc67d306f None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:29.747608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ce082-106a-4ab1-81a9-491cc67d306f None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:29.780770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f51bc-4d8f-4c2f-aaae-1a6eff4da41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:29.781405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f51bc-4d8f-4c2f-aaae-1a6eff4da41b None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:29.781690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f51bc-4d8f-4c2f-aaae-1a6eff4da41b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:29.781967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f51bc-4d8f-4c2f-aaae-1a6eff4da41b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:29.782818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f51bc-4d8f-4c2f-aaae-1a6eff4da41b None admin] RBAC: Adding query filter params (role.id=433478cc318a467c931f13a757f161fa, scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d, user.id=a786a95c48c348dbba269e0e33265271) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:29.783134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f51bc-4d8f-4c2f-aaae-1a6eff4da41b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:29.784549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f51bc-4d8f-4c2f-aaae-1a6eff4da41b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:29.790226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 637/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Jun 15 15:40:29 2022] GET /identity/v3/role_assignments?user.id=a786a95c48c348dbba269e0e33265271&scope.project.id=9aa8cd5804554314b5b0a61cb3c8ec9d&role.id=433478cc318a467c931f13a757f161fa => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:31.923116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce866b46-e209-4a9b-84b8-94b36941785d None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:31.923660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce866b46-e209-4a9b-84b8-94b36941785d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:31.923660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce866b46-e209-4a9b-84b8-94b36941785d None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:31.925025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 651/1288] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:31.930130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077df6b6-8577-4fcf-9fef-6ace7954d07c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:31.930526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077df6b6-8577-4fcf-9fef-6ace7954d07c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:31.930692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077df6b6-8577-4fcf-9fef-6ace7954d07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:31.947361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-077df6b6-8577-4fcf-9fef-6ace7954d07c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:31.948529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-077df6b6-8577-4fcf-9fef-6ace7954d07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:31.976960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:32.015298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 638/1289] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:32.045450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce866b46-e209-4a9b-84b8-94b36941785d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:32.142866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce866b46-e209-4a9b-84b8-94b36941785d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:32.153554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:32.177001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72500e81-76fb-4d96-8f7f-de5d173325ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:32.177733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72500e81-76fb-4d96-8f7f-de5d173325ff None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:32.178015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72500e81-76fb-4d96-8f7f-de5d173325ff None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:32.178298 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72500e81-76fb-4d96-8f7f-de5d173325ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:32.178975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72500e81-76fb-4d96-8f7f-de5d173325ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:32.180341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72500e81-76fb-4d96-8f7f-de5d173325ff None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-72500e81-76fb-4d96-8f7f-de5d173325ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:32.192001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:32.193138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:32.194385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 652/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:40:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:32.199657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077df6b6-8577-4fcf-9fef-6ace7954d07c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:32.303266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077df6b6-8577-4fcf-9fef-6ace7954d07c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:32.335976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67712b6d-ca9a-407b-8a63-3cc4874e5e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:32.336660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67712b6d-ca9a-407b-8a63-3cc4874e5e26 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:32.336943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67712b6d-ca9a-407b-8a63-3cc4874e5e26 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:32.337788 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67712b6d-ca9a-407b-8a63-3cc4874e5e26 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:32.338490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67712b6d-ca9a-407b-8a63-3cc4874e5e26 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:32.338828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67712b6d-ca9a-407b-8a63-3cc4874e5e26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:32.340183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67712b6d-ca9a-407b-8a63-3cc4874e5e26 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:32.344931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 639/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Jun 15 15:40:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:32.350661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72500e81-76fb-4d96-8f7f-de5d173325ff None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:32.448664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72500e81-76fb-4d96-8f7f-de5d173325ff None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:32.481402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac949b-5a80-4bb1-a98e-efc4609afdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:32.482053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac949b-5a80-4bb1-a98e-efc4609afdca None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:32.482336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac949b-5a80-4bb1-a98e-efc4609afdca None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:32.482610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac949b-5a80-4bb1-a98e-efc4609afdca None admin] PATH_INFO: `/v3/services` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:32.483341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac949b-5a80-4bb1-a98e-efc4609afdca None admin] RBAC: Adding query filter params (type=placement) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:32.483649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac949b-5a80-4bb1-a98e-efc4609afdca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:32.485006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac949b-5a80-4bb1-a98e-efc4609afdca None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:32.489341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 653/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Jun 15 15:40:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:34.569494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb70b1-91a7-425c-925b-97e581ac196d None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:34.570032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb70b1-91a7-425c-925b-97e581ac196d None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:34.570032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb70b1-91a7-425c-925b-97e581ac196d None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:34.571422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 640/1293] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:34.576461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd136d94-e88e-4703-9b60-db0321dcb7e1 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:34.576829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd136d94-e88e-4703-9b60-db0321dcb7e1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:34.577023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd136d94-e88e-4703-9b60-db0321dcb7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:34.593318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-fd136d94-e88e-4703-9b60-db0321dcb7e1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:34.594498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fd136d94-e88e-4703-9b60-db0321dcb7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:34.623760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:34.660030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 654/1294] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:34.683499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb70b1-91a7-425c-925b-97e581ac196d None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:34.713209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:34.783466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb70b1-91a7-425c-925b-97e581ac196d None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:34.819588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c51b9-7379-4778-b201-2d8ade198a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:34.820241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c51b9-7379-4778-b201-2d8ade198a3b None admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:34.820526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c51b9-7379-4778-b201-2d8ade198a3b None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:34.820805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c51b9-7379-4778-b201-2d8ade198a3b None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:34.822196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c51b9-7379-4778-b201-2d8ade198a3b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:34.823593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c51b9-7379-4778-b201-2d8ade198a3b None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:34.837989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 641/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Jun 15 15:40:34 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:40:36.952083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed766bc-c1d3-4b4e-a25b-0654e0294caf None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:36.952610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed766bc-c1d3-4b4e-a25b-0654e0294caf None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:36.952728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed766bc-c1d3-4b4e-a25b-0654e0294caf None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:36.954022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 655/1296] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:36.959035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef6f57-7f17-47dc-a28c-13f4b7d49b1c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:36.959466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef6f57-7f17-47dc-a28c-13f4b7d49b1c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:36.959592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef6f57-7f17-47dc-a28c-13f4b7d49b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:36.975718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d9ef6f57-7f17-47dc-a28c-13f4b7d49b1c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:36.976874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d9ef6f57-7f17-47dc-a28c-13f4b7d49b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:37.046164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 642/1297] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:37.068566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed766bc-c1d3-4b4e-a25b-0654e0294caf None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:37.165843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed766bc-c1d3-4b4e-a25b-0654e0294caf None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:37.197653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc3100-16dc-4f75-bb02-a49e2716b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:37.198443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc3100-16dc-4f75-bb02-a49e2716b08f None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:37.198771 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc3100-16dc-4f75-bb02-a49e2716b08f None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:37.199053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc3100-16dc-4f75-bb02-a49e2716b08f None admin] PATH_INFO: `/v3/services/placement` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:37.199696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc3100-16dc-4f75-bb02-a49e2716b08f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:37.201059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc3100-16dc-4f75-bb02-a49e2716b08f None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-b2fc3100-16dc-4f75-bb02-a49e2716b08f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:37.214815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:37.216133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:37.217358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 656/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:40:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:37.222796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef6f57-7f17-47dc-a28c-13f4b7d49b1c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:37.244499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:37.326315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef6f57-7f17-47dc-a28c-13f4b7d49b1c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:37.359970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e164a-615b-4625-a7c7-eaafc9ed8693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:37.360626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e164a-615b-4625-a7c7-eaafc9ed8693 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:37.360911 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e164a-615b-4625-a7c7-eaafc9ed8693 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:37.361190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e164a-615b-4625-a7c7-eaafc9ed8693 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:37.361887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e164a-615b-4625-a7c7-eaafc9ed8693 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:37.362197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e164a-615b-4625-a7c7-eaafc9ed8693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:37.363663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e164a-615b-4625-a7c7-eaafc9ed8693 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:37.368741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 643/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Jun 15 15:40:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:37.374806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc3100-16dc-4f75-bb02-a49e2716b08f None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:37.405839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:37.476904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc3100-16dc-4f75-bb02-a49e2716b08f None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:37.508183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6b91b-312e-4806-9d79-9dbcfba4f295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:37.509343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6b91b-312e-4806-9d79-9dbcfba4f295 None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:37.509617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6b91b-312e-4806-9d79-9dbcfba4f295 None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:37.509885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6b91b-312e-4806-9d79-9dbcfba4f295 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:37.510660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6b91b-312e-4806-9d79-9dbcfba4f295 None admin] RBAC: Adding query filter params (interface=public, service_id=67945c8c0e8e4c82ae0de39e47eaea89, region_id=RegionOne) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:37.510995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6b91b-312e-4806-9d79-9dbcfba4f295 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:37.512611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6b91b-312e-4806-9d79-9dbcfba4f295 None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:37.517445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 657/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Jun 15 15:40:37 2022] GET /identity/v3/endpoints?service_id=67945c8c0e8e4c82ae0de39e47eaea89&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:37.523047 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e164a-615b-4625-a7c7-eaafc9ed8693 None admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:37.604518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:37.622624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e164a-615b-4625-a7c7-eaafc9ed8693 None admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:37.656446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ccddb-397e-4b95-85ca-560da57234bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:37.657095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ccddb-397e-4b95-85ca-560da57234bd None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:37.657375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ccddb-397e-4b95-85ca-560da57234bd None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:37.657649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ccddb-397e-4b95-85ca-560da57234bd None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:37.658321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ccddb-397e-4b95-85ca-560da57234bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:37.659727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ccddb-397e-4b95-85ca-560da57234bd None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:37.666425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 644/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Jun 15 15:40:37 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:40:39.754674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5296a6d-6e55-4fa7-aa70-f141c5a44ae8 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:39.755291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5296a6d-6e55-4fa7-aa70-f141c5a44ae8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:39.755291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5296a6d-6e55-4fa7-aa70-f141c5a44ae8 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:39.756550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 658/1302] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:40:39.761564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc5703-c7fc-4b79-bfa3-d85040bd6e97 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:39.762229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc5703-c7fc-4b79-bfa3-d85040bd6e97 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:39.762375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc5703-c7fc-4b79-bfa3-d85040bd6e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:39.778444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e4dc5703-c7fc-4b79-bfa3-d85040bd6e97 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:40:39.779629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e4dc5703-c7fc-4b79-bfa3-d85040bd6e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:40:39.844839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 645/1303] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:40:39 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 15:40:39.867672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5296a6d-6e55-4fa7-aa70-f141c5a44ae8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:39.960054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:39.967028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5296a6d-6e55-4fa7-aa70-f141c5a44ae8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:39.999868 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905533c0-c432-498b-856a-9e2cfe01c96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:40.000638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905533c0-c432-498b-856a-9e2cfe01c96b None admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:40.000918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905533c0-c432-498b-856a-9e2cfe01c96b None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:40.001191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905533c0-c432-498b-856a-9e2cfe01c96b None admin] PATH_INFO: `/v3/services/placement` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:40.001834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905533c0-c432-498b-856a-9e2cfe01c96b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:40.003222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905533c0-c432-498b-856a-9e2cfe01c96b None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application [None req-905533c0-c432-498b-856a-9e2cfe01c96b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 15:40:40.015116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 15:40:40.016265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application with Lock( Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self._enter() Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application return self.creator() Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application created_value = creator( Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: ERROR keystone.server.flask.application Jun 15 15:40:40.017289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 659/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Jun 15 15:40:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 15:40:40.022030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc5703-c7fc-4b79-bfa3-d85040bd6e97 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:40.096810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:40.119643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc5703-c7fc-4b79-bfa3-d85040bd6e97 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:40.159925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871273a-1b21-4926-8826-a8a177f7bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:40.160861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871273a-1b21-4926-8826-a8a177f7bce7 None admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:40.161145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871273a-1b21-4926-8826-a8a177f7bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:40.161423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871273a-1b21-4926-8826-a8a177f7bce7 None admin] PATH_INFO: `/v3/services` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:40.162124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871273a-1b21-4926-8826-a8a177f7bce7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:40:40.162429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871273a-1b21-4926-8826-a8a177f7bce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:40.163812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871273a-1b21-4926-8826-a8a177f7bce7 None admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:40.169467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 646/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Jun 15 15:40:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:40:40.175186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905533c0-c432-498b-856a-9e2cfe01c96b None admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:40:40.269415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905533c0-c432-498b-856a-9e2cfe01c96b None admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:40:40.301697 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affad9d-5fb6-4be5-aa89-886785cfbf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:40:40.302324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affad9d-5fb6-4be5-aa89-886785cfbf2a None admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:40:40.302600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affad9d-5fb6-4be5-aa89-886785cfbf2a None admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:40:40.302899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affad9d-5fb6-4be5-aa89-886785cfbf2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:40:40.303622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affad9d-5fb6-4be5-aa89-886785cfbf2a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:40:40.304963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affad9d-5fb6-4be5-aa89-886785cfbf2a None admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:40:40.321962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 660/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Jun 15 15:40:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:40:50.166914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=30 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:40:50.318344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:42.882421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed01a2-59fb-491f-a2a2-98732bc26fe8 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:42.883006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed01a2-59fb-491f-a2a2-98732bc26fe8 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:42.883006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed01a2-59fb-491f-a2a2-98732bc26fe8 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:42.889665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a0deb-c6a0-4d6a-a446-a388a75dfbff None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:42.890102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a0deb-c6a0-4d6a-a446-a388a75dfbff None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:42.890194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a0deb-c6a0-4d6a-a446-a388a75dfbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:42.893674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 647/1307] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:41:42 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:41:43.031227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-3c5a0deb-c6a0-4d6a-a446-a388a75dfbff None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:41:43.032886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3c5a0deb-c6a0-4d6a-a446-a388a75dfbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:43.145609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 661/1308] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:41:43.220155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2f783-9a2f-4615-95ca-daae611cc5e7 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:43.220449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2f783-9a2f-4615-95ca-daae611cc5e7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:43.220728 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2f783-9a2f-4615-95ca-daae611cc5e7 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:43.222491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 648/1309] 10.222.0.62 () {54 vars in 964 bytes} [Wed Jun 15 15:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:41:43.229340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b60ddc-c94c-4b69-bc49-b4b73bbe12df None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:43.229632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b60ddc-c94c-4b69-bc49-b4b73bbe12df None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:43.229912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b60ddc-c94c-4b69-bc49-b4b73bbe12df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:43.382400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-37b60ddc-c94c-4b69-bc49-b4b73bbe12df None None] MFA Rules not processed for user `bf1915b9919442d692bcedf25f53320c`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:41:43.383661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-37b60ddc-c94c-4b69-bc49-b4b73bbe12df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:43.423368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:43.530287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 662/1310] 10.222.0.62 () {58 vars in 1061 bytes} [Wed Jun 15 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:41:43.539607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2f783-9a2f-4615-95ca-daae611cc5e7 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:41:43.541468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fef2f783-9a2f-4615-95ca-daae611cc5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:43.703374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2f783-9a2f-4615-95ca-daae611cc5e7 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:41:43.805779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5a462-7f55-49d6-97ff-d05c1d3329de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:41:43.806483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5a462-7f55-49d6-97ff-d05c1d3329de service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:43.806793 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5a462-7f55-49d6-97ff-d05c1d3329de service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:43.807080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5a462-7f55-49d6-97ff-d05c1d3329de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:43.808977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-93c5a462-7f55-49d6-97ff-d05c1d3329de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:43.878579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:43.958061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5a462-7f55-49d6-97ff-d05c1d3329de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:41:43.959576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5a462-7f55-49d6-97ff-d05c1d3329de service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:41:43.983951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 649/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:41:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:41:46.436022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f14249-9d48-4df9-ab26-56a12528866d None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:46.436561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f14249-9d48-4df9-ab26-56a12528866d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:46.436561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f14249-9d48-4df9-ab26-56a12528866d None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:46.437930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 663/1312] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:41:46.443508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4ec98-32c4-4699-9727-9b214d18e423 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:46.443953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4ec98-32c4-4699-9727-9b214d18e423 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:46.444066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4ec98-32c4-4699-9727-9b214d18e423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:46.529283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-dff4ec98-32c4-4699-9727-9b214d18e423 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:41:46.530529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-dff4ec98-32c4-4699-9727-9b214d18e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:46.627920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 650/1313] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:41:46.671460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f14249-9d48-4df9-ab26-56a12528866d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:41:46.673145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-67f14249-9d48-4df9-ab26-56a12528866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:46.777466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f14249-9d48-4df9-ab26-56a12528866d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:41:46.851742 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:46.854951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfae677-b515-4df2-a36e-81d36cc636de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:41:46.855766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfae677-b515-4df2-a36e-81d36cc636de service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:46.856291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfae677-b515-4df2-a36e-81d36cc636de service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:46.856595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfae677-b515-4df2-a36e-81d36cc636de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:46.858255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-abfae677-b515-4df2-a36e-81d36cc636de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:46.964178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfae677-b515-4df2-a36e-81d36cc636de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:41:46.965602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfae677-b515-4df2-a36e-81d36cc636de service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:41:46.983467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 664/1314] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:41:49.385780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686807a-79f7-4cb6-92f6-1d5c15e42236 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:49.386318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686807a-79f7-4cb6-92f6-1d5c15e42236 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:49.386318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686807a-79f7-4cb6-92f6-1d5c15e42236 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:49.387743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 651/1315] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:41:49.392915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c57a97-2949-4c85-ab4d-8a69462c5e94 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:49.393291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c57a97-2949-4c85-ab4d-8a69462c5e94 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:49.393480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c57a97-2949-4c85-ab4d-8a69462c5e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:49.456338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-c3c57a97-2949-4c85-ab4d-8a69462c5e94 None None] MFA Rules not processed for user `4042a3d4d3d8483b9f9a6f1c750f1360`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:41:49.457562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c3c57a97-2949-4c85-ab4d-8a69462c5e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:49.515849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:49.560143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 665/1316] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:41:49.599305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686807a-79f7-4cb6-92f6-1d5c15e42236 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:41:49.600851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1686807a-79f7-4cb6-92f6-1d5c15e42236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:49.610321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:49.702513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686807a-79f7-4cb6-92f6-1d5c15e42236 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:41:49.773913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7ddaf-2b52-4bf2-af6d-d9b07555b6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:41:49.774558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7ddaf-2b52-4bf2-af6d-d9b07555b6fa service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:49.774880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7ddaf-2b52-4bf2-af6d-d9b07555b6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:49.775157 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7ddaf-2b52-4bf2-af6d-d9b07555b6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:49.776791 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fdc7ddaf-2b52-4bf2-af6d-d9b07555b6fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:49.839325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:49.884794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7ddaf-2b52-4bf2-af6d-d9b07555b6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:41:49.886214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7ddaf-2b52-4bf2-af6d-d9b07555b6fa service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:41:49.904581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 652/1317] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:41:49 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:41:52.555415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399eb00-6154-4222-a89f-51c3da2d8168 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:52.555936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399eb00-6154-4222-a89f-51c3da2d8168 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:52.556049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399eb00-6154-4222-a89f-51c3da2d8168 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:52.557339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 666/1318] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:41:52.562508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e213e1-2fc0-4b19-ae9b-b2eb36c8f958 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:52.562957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e213e1-2fc0-4b19-ae9b-b2eb36c8f958 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:52.563087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e213e1-2fc0-4b19-ae9b-b2eb36c8f958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:52.625912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-77e213e1-2fc0-4b19-ae9b-b2eb36c8f958 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:41:52.627143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-77e213e1-2fc0-4b19-ae9b-b2eb36c8f958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:52.729080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 653/1319] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:41:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:41:52.770442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399eb00-6154-4222-a89f-51c3da2d8168 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:41:52.772058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2399eb00-6154-4222-a89f-51c3da2d8168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:52.874185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399eb00-6154-4222-a89f-51c3da2d8168 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:41:52.945313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6ed52-8cff-477d-bd52-9c4c3aa20365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:41:52.945949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6ed52-8cff-477d-bd52-9c4c3aa20365 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:52.946232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6ed52-8cff-477d-bd52-9c4c3aa20365 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:52.946508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6ed52-8cff-477d-bd52-9c4c3aa20365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:52.948187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-88e6ed52-8cff-477d-bd52-9c4c3aa20365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:52.955378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:53.055373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6ed52-8cff-477d-bd52-9c4c3aa20365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:41:53.056745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6ed52-8cff-477d-bd52-9c4c3aa20365 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:41:53.074341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 667/1320] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:41:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:41:55.823686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9d26-698c-41f9-b009-a3b99a3f57f0 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:55.824236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9d26-698c-41f9-b009-a3b99a3f57f0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:55.824236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9d26-698c-41f9-b009-a3b99a3f57f0 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:55.825559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 654/1321] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:41:55.830702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b82dc-6ef9-4e2b-b725-0118303e63ce None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:55.831168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b82dc-6ef9-4e2b-b725-0118303e63ce None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:55.831277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b82dc-6ef9-4e2b-b725-0118303e63ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:55.902556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-eb2b82dc-6ef9-4e2b-b725-0118303e63ce None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:41:55.903847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-eb2b82dc-6ef9-4e2b-b725-0118303e63ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:55.965355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:56.007389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 668/1322] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:41:56.043992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e9d26-698c-41f9-b009-a3b99a3f57f0 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:41:56.045520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-418e9d26-698c-41f9-b009-a3b99a3f57f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:56.065736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:56.152257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e9d26-698c-41f9-b009-a3b99a3f57f0 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:41:56.227357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c040423-34f7-49a6-99db-cc61682ab0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:41:56.228268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c040423-34f7-49a6-99db-cc61682ab0ef service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:56.228552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c040423-34f7-49a6-99db-cc61682ab0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:56.228828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c040423-34f7-49a6-99db-cc61682ab0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:56.230494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2c040423-34f7-49a6-99db-cc61682ab0ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:56.301054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:56.339456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c040423-34f7-49a6-99db-cc61682ab0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:41:56.340885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c040423-34f7-49a6-99db-cc61682ab0ef service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:41:56.359042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 655/1323] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:41:58.544540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbeeaa-99f0-4bfc-a662-b419de6fe3a5 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:58.545123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbeeaa-99f0-4bfc-a662-b419de6fe3a5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:58.545123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbeeaa-99f0-4bfc-a662-b419de6fe3a5 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:58.546546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 669/1324] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:41:58.551689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f341e-c599-4ac9-b11f-b4fcab0da1cd None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:58.552122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f341e-c599-4ac9-b11f-b4fcab0da1cd None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:58.552223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f341e-c599-4ac9-b11f-b4fcab0da1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:58.617680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-136f341e-c599-4ac9-b11f-b4fcab0da1cd None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:41:58.618922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-136f341e-c599-4ac9-b11f-b4fcab0da1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:58.714792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 656/1325] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:41:58.748896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbeeaa-99f0-4bfc-a662-b419de6fe3a5 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:41:58.750480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-addbeeaa-99f0-4bfc-a662-b419de6fe3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:58.860359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbeeaa-99f0-4bfc-a662-b419de6fe3a5 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:41:58.933681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61adac-9b31-4d2f-a13e-b0ef4b5e1ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:41:58.934326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61adac-9b31-4d2f-a13e-b0ef4b5e1ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:41:58.934607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61adac-9b31-4d2f-a13e-b0ef4b5e1ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:41:58.934914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61adac-9b31-4d2f-a13e-b0ef4b5e1ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:41:58.936551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ca61adac-9b31-4d2f-a13e-b0ef4b5e1ba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:41:58.953431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:41:59.054840 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61adac-9b31-4d2f-a13e-b0ef4b5e1ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:41:59.056294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61adac-9b31-4d2f-a13e-b0ef4b5e1ba7 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:41:59.074265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 670/1326] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:01.233854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b24bc2-8627-4784-b90d-cc38ef34ff72 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:01.234396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b24bc2-8627-4784-b90d-cc38ef34ff72 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:01.234396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b24bc2-8627-4784-b90d-cc38ef34ff72 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:01.235818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 657/1327] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:01.241416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13188a1e-cbe5-45e5-8965-18fdf1802dd7 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:01.241803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13188a1e-cbe5-45e5-8965-18fdf1802dd7 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:01.241973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13188a1e-cbe5-45e5-8965-18fdf1802dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:01.305790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-13188a1e-cbe5-45e5-8965-18fdf1802dd7 None None] MFA Rules not processed for user `4042a3d4d3d8483b9f9a6f1c750f1360`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:01.307009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-13188a1e-cbe5-45e5-8965-18fdf1802dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:01.372151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:01.406789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 671/1328] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:01 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:42:01.457601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2abfb0-6dde-452a-90ee-76591c060ce8 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:01.457895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2abfb0-6dde-452a-90ee-76591c060ce8 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:01.458176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2abfb0-6dde-452a-90ee-76591c060ce8 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:01.459597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 658/1329] 10.222.0.62 () {54 vars in 964 bytes} [Wed Jun 15 15:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:01.465422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ecd84-61f5-4bb1-bc4f-88326dac7d63 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:01.465709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ecd84-61f5-4bb1-bc4f-88326dac7d63 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:01.465983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ecd84-61f5-4bb1-bc4f-88326dac7d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:01.539369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-0d7ecd84-61f5-4bb1-bc4f-88326dac7d63 None None] MFA Rules not processed for user `bf1915b9919442d692bcedf25f53320c`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:01.540614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0d7ecd84-61f5-4bb1-bc4f-88326dac7d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:01.636568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 672/1330] 10.222.0.62 () {58 vars in 1061 bytes} [Wed Jun 15 15:42:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:01.645781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2abfb0-6dde-452a-90ee-76591c060ce8 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:01.647378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3d2abfb0-6dde-452a-90ee-76591c060ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:01.680459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:01.754025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2abfb0-6dde-452a-90ee-76591c060ce8 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:01.826098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3ef22-baff-44a5-8d04-730966ce3fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:01.826675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3ef22-baff-44a5-8d04-730966ce3fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:01.827002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3ef22-baff-44a5-8d04-730966ce3fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:01.827349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3ef22-baff-44a5-8d04-730966ce3fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:01.829035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-00f3ef22-baff-44a5-8d04-730966ce3fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:01.906012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:01.938169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3ef22-baff-44a5-8d04-730966ce3fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:01.939651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3ef22-baff-44a5-8d04-730966ce3fe4 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:01.960175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 659/1331] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:04.688763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26c4aa-2511-48a6-9bbd-ddbe99b19368 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:04.689312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26c4aa-2511-48a6-9bbd-ddbe99b19368 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:04.689312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26c4aa-2511-48a6-9bbd-ddbe99b19368 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:04.690769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 673/1332] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:04.696283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff090693-e419-4120-afbe-39865c59f4dc None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:04.696734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff090693-e419-4120-afbe-39865c59f4dc None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:04.696734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff090693-e419-4120-afbe-39865c59f4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:04.763445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ff090693-e419-4120-afbe-39865c59f4dc None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:04.764569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ff090693-e419-4120-afbe-39865c59f4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:04.885914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 660/1333] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:04.923078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26c4aa-2511-48a6-9bbd-ddbe99b19368 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:04.924690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ac26c4aa-2511-48a6-9bbd-ddbe99b19368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:04.998279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:05.032665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26c4aa-2511-48a6-9bbd-ddbe99b19368 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:05.104002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c75158-37c9-40b3-a7d7-09cf9719ae6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:05.104639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c75158-37c9-40b3-a7d7-09cf9719ae6a service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:05.104920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c75158-37c9-40b3-a7d7-09cf9719ae6a service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:05.105192 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c75158-37c9-40b3-a7d7-09cf9719ae6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:05.106868 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e1c75158-37c9-40b3-a7d7-09cf9719ae6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:05.213287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c75158-37c9-40b3-a7d7-09cf9719ae6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:05.214720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c75158-37c9-40b3-a7d7-09cf9719ae6a service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:05.225256 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:05.233703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 674/1334] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:07.384750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838df73-dee9-46b5-9c64-cd10cbaebe7b None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:07.385318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838df73-dee9-46b5-9c64-cd10cbaebe7b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:07.385318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838df73-dee9-46b5-9c64-cd10cbaebe7b None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:07.386685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 661/1335] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:07.391741 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c3b6b-74a5-4e35-b8e3-c5064d097040 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:07.392136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c3b6b-74a5-4e35-b8e3-c5064d097040 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:07.392305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c3b6b-74a5-4e35-b8e3-c5064d097040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:07.457910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-cb7c3b6b-74a5-4e35-b8e3-c5064d097040 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:07.459165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cb7c3b6b-74a5-4e35-b8e3-c5064d097040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:07.554678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 675/1336] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:07.593792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838df73-dee9-46b5-9c64-cd10cbaebe7b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:07.595407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1838df73-dee9-46b5-9c64-cd10cbaebe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:07.635780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:07.698540 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838df73-dee9-46b5-9c64-cd10cbaebe7b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:07.771244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27410f-8b2f-4f2e-af68-23d8221eda92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:07.771891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27410f-8b2f-4f2e-af68-23d8221eda92 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:07.772178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27410f-8b2f-4f2e-af68-23d8221eda92 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:07.772475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27410f-8b2f-4f2e-af68-23d8221eda92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:07.774102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2a27410f-8b2f-4f2e-af68-23d8221eda92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:07.856004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:07.881548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27410f-8b2f-4f2e-af68-23d8221eda92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:07.883011 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27410f-8b2f-4f2e-af68-23d8221eda92 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:07.901832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 662/1337] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:10.327870 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89753a5f-5497-4ee9-82a5-337c47b37a29 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:10.328439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89753a5f-5497-4ee9-82a5-337c47b37a29 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:10.328439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89753a5f-5497-4ee9-82a5-337c47b37a29 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:10.329780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 676/1338] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:10.335552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db914f8-fc5d-43ac-a390-dab5c26f111e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:10.335976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db914f8-fc5d-43ac-a390-dab5c26f111e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:10.336224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db914f8-fc5d-43ac-a390-dab5c26f111e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:10.400019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-3db914f8-fc5d-43ac-a390-dab5c26f111e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:10.401217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3db914f8-fc5d-43ac-a390-dab5c26f111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:10.502903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 663/1339] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:10.538668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89753a5f-5497-4ee9-82a5-337c47b37a29 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:10.540230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-89753a5f-5497-4ee9-82a5-337c47b37a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:10.614982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:10.643951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89753a5f-5497-4ee9-82a5-337c47b37a29 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:10.714006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e07a1-7d17-48d4-ad72-a83a1ecd3caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:10.714637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e07a1-7d17-48d4-ad72-a83a1ecd3caf service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:10.714942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e07a1-7d17-48d4-ad72-a83a1ecd3caf service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:10.715213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e07a1-7d17-48d4-ad72-a83a1ecd3caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:10.716823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-658e07a1-7d17-48d4-ad72-a83a1ecd3caf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:10.822860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e07a1-7d17-48d4-ad72-a83a1ecd3caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:10.824338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e07a1-7d17-48d4-ad72-a83a1ecd3caf service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:10.841801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 677/1340] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:15.009381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322df269-7a81-432a-8f2d-5cfbc00bda73 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:15.009964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322df269-7a81-432a-8f2d-5cfbc00bda73 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:15.009964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322df269-7a81-432a-8f2d-5cfbc00bda73 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:15.011326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 664/1341] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:15.016328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56b08a-41a0-45b9-8230-6cd4e1e4269f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:15.016747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56b08a-41a0-45b9-8230-6cd4e1e4269f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:15.017042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56b08a-41a0-45b9-8230-6cd4e1e4269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:15.023876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:15.087403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-1c56b08a-41a0-45b9-8230-6cd4e1e4269f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:15.088505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1c56b08a-41a0-45b9-8230-6cd4e1e4269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:15.197316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 678/1342] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:15.238671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322df269-7a81-432a-8f2d-5cfbc00bda73 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:15.240842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-322df269-7a81-432a-8f2d-5cfbc00bda73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:15.291620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:15.338403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322df269-7a81-432a-8f2d-5cfbc00bda73 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:15.341988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e4a30-4343-474f-83a5-a2918dabc0a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:15.342640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e4a30-4343-474f-83a5-a2918dabc0a2 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:15.342955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e4a30-4343-474f-83a5-a2918dabc0a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:15.343238 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e4a30-4343-474f-83a5-a2918dabc0a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:15.352384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e4a30-4343-474f-83a5-a2918dabc0a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:15.353807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e4a30-4343-474f-83a5-a2918dabc0a2 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:15.364058 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 665/1343] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:18.121350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a4641-e33d-4997-9865-6796d8571257 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:18.121917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a4641-e33d-4997-9865-6796d8571257 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:18.121917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a4641-e33d-4997-9865-6796d8571257 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:18.123332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 679/1344] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:18.129347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe1c96-49e0-46ef-8cc3-7799d0d32a58 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:18.129855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe1c96-49e0-46ef-8cc3-7799d0d32a58 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:18.129941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe1c96-49e0-46ef-8cc3-7799d0d32a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:18.160391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-dcbe1c96-49e0-46ef-8cc3-7799d0d32a58 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:18.161635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-dcbe1c96-49e0-46ef-8cc3-7799d0d32a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:18.254393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 666/1345] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:18.296086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a4641-e33d-4997-9865-6796d8571257 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:18.297873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-585a4641-e33d-4997-9865-6796d8571257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:18.311393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a4641-e33d-4997-9865-6796d8571257 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:18.314975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965c07f-7b91-4083-8f4f-2a8336a9a8f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:18.315705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965c07f-7b91-4083-8f4f-2a8336a9a8f2 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:18.315988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965c07f-7b91-4083-8f4f-2a8336a9a8f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:18.316262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965c07f-7b91-4083-8f4f-2a8336a9a8f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:18.325135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965c07f-7b91-4083-8f4f-2a8336a9a8f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:18.326569 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965c07f-7b91-4083-8f4f-2a8336a9a8f2 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:18.336568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 680/1346] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:22.654238 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41800c-5d59-4d09-84c4-6003770a2b23 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:22.654794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41800c-5d59-4d09-84c4-6003770a2b23 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:22.654991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41800c-5d59-4d09-84c4-6003770a2b23 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:22.656364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 667/1347] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:22.662145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a0120-762c-4330-ad4c-729fd7963e55 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:22.662562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a0120-762c-4330-ad4c-729fd7963e55 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:22.662706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a0120-762c-4330-ad4c-729fd7963e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:22.680406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9d8a0120-762c-4330-ad4c-729fd7963e55 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:22.681602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9d8a0120-762c-4330-ad4c-729fd7963e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:22.689559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 681/1348] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:22.727671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41800c-5d59-4d09-84c4-6003770a2b23 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:22.738648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41800c-5d59-4d09-84c4-6003770a2b23 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:22.742167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415eb90-3d8f-4bc6-a1f1-dd758769d578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:22.742838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415eb90-3d8f-4bc6-a1f1-dd758769d578 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:22.743123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415eb90-3d8f-4bc6-a1f1-dd758769d578 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:22.743406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415eb90-3d8f-4bc6-a1f1-dd758769d578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:22.752981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415eb90-3d8f-4bc6-a1f1-dd758769d578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:22.754469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415eb90-3d8f-4bc6-a1f1-dd758769d578 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:22.764963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 668/1349] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:42:27.791858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7b0d9-cba9-49d8-8bae-0ba932b8b238 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:27.792368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7b0d9-cba9-49d8-8bae-0ba932b8b238 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:27.792509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7b0d9-cba9-49d8-8bae-0ba932b8b238 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:27.794363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 682/1350] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:27.799189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad94e9-2a8b-49c0-ae6a-ad1c41ddbb3e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:27.800008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad94e9-2a8b-49c0-ae6a-ad1c41ddbb3e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:27.800506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad94e9-2a8b-49c0-ae6a-ad1c41ddbb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:27.871845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9aad94e9-2a8b-49c0-ae6a-ad1c41ddbb3e None None] MFA Rules not processed for user `a786a95c48c348dbba269e0e33265271`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:27.873668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9aad94e9-2a8b-49c0-ae6a-ad1c41ddbb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:27.904332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:27.958676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 669/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:42:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:32.172850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f2c4-1797-4713-8a93-634cc18b791c None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:32.173809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f2c4-1797-4713-8a93-634cc18b791c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:32.174290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f2c4-1797-4713-8a93-634cc18b791c None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:32.176195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 683/1352] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:32.181491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f24ab-e368-49fc-bd99-fcccb57bc998 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:32.181945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f24ab-e368-49fc-bd99-fcccb57bc998 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:32.182182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f24ab-e368-49fc-bd99-fcccb57bc998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:32.202944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-785f24ab-e368-49fc-bd99-fcccb57bc998 None None] MFA Rules not processed for user `a786a95c48c348dbba269e0e33265271`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:32.204794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-785f24ab-e368-49fc-bd99-fcccb57bc998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:32.213636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 670/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:42:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:32.675691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=72 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:33.707564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb6bfb-b193-4c91-aae2-8339325c9532 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:33.708157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb6bfb-b193-4c91-aae2-8339325c9532 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:33.708157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb6bfb-b193-4c91-aae2-8339325c9532 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:33.709395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 684/1354] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:33.714469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42822d82-4332-4757-95a4-d79d059e16c2 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:33.714906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42822d82-4332-4757-95a4-d79d059e16c2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:33.715057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42822d82-4332-4757-95a4-d79d059e16c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:33.733869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-42822d82-4332-4757-95a4-d79d059e16c2 None None] MFA Rules not processed for user `a786a95c48c348dbba269e0e33265271`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:33.735091 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-42822d82-4332-4757-95a4-d79d059e16c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:33.743460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 671/1355] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:42:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:42:37.519877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e3bba-5da8-40db-ba6e-c911f6e2993a None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:37.520430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e3bba-5da8-40db-ba6e-c911f6e2993a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:37.520430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e3bba-5da8-40db-ba6e-c911f6e2993a None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:37.521978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 685/1356] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:37.527072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435cf27-c997-41f9-a7d8-3436b8e1296e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:37.527498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435cf27-c997-41f9-a7d8-3436b8e1296e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:37.527830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435cf27-c997-41f9-a7d8-3436b8e1296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:37.546469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-3435cf27-c997-41f9-a7d8-3436b8e1296e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:37.548494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3435cf27-c997-41f9-a7d8-3436b8e1296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:37.556732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 672/1357] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:37.609259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a0e6a-f5eb-4808-8fcc-2748fc31965a None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:37.610402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a0e6a-f5eb-4808-8fcc-2748fc31965a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:37.610947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a0e6a-f5eb-4808-8fcc-2748fc31965a None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:37.612819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 686/1358] 10.222.0.62 () {54 vars in 961 bytes} [Wed Jun 15 15:42:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:37.626889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5d9b0-a047-43cc-af6d-d5a21dbb8ea4 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:37.628491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5d9b0-a047-43cc-af6d-d5a21dbb8ea4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:37.629614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5d9b0-a047-43cc-af6d-d5a21dbb8ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:37.683120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-02e5d9b0-a047-43cc-af6d-d5a21dbb8ea4 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:37.684391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-02e5d9b0-a047-43cc-af6d-d5a21dbb8ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:37.767033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 673/1359] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Jun 15 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:37.774824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a0e6a-f5eb-4808-8fcc-2748fc31965a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:37.790416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a0e6a-f5eb-4808-8fcc-2748fc31965a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:37.796378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c40a1-2c22-41bc-9a8b-46e3c9f1599e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:37.797727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c40a1-2c22-41bc-9a8b-46e3c9f1599e service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:37.798079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c40a1-2c22-41bc-9a8b-46e3c9f1599e service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:37.798447 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c40a1-2c22-41bc-9a8b-46e3c9f1599e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:37.807366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c40a1-2c22-41bc-9a8b-46e3c9f1599e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:37.809244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c40a1-2c22-41bc-9a8b-46e3c9f1599e service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:37.819827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 687/1360] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:40.143482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ce14-df0f-40a7-a1b4-6fea51b12bb6 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:40.144059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ce14-df0f-40a7-a1b4-6fea51b12bb6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:40.144059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ce14-df0f-40a7-a1b4-6fea51b12bb6 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:40.145596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 674/1361] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:40.150967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ca9e1-25c7-4104-8e41-06785e77030c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:40.151522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ca9e1-25c7-4104-8e41-06785e77030c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:40.151522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ca9e1-25c7-4104-8e41-06785e77030c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:40.171435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-275ca9e1-25c7-4104-8e41-06785e77030c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:40.173300 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-275ca9e1-25c7-4104-8e41-06785e77030c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:40.181928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 688/1362] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:40.228509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33ce14-df0f-40a7-a1b4-6fea51b12bb6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:40.243343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33ce14-df0f-40a7-a1b4-6fea51b12bb6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:40.248782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb095a-9c7e-4844-9e64-20b996c97942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:40.249419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb095a-9c7e-4844-9e64-20b996c97942 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:40.249704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb095a-9c7e-4844-9e64-20b996c97942 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:40.250617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb095a-9c7e-4844-9e64-20b996c97942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:40.260886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb095a-9c7e-4844-9e64-20b996c97942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:40.262481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb095a-9c7e-4844-9e64-20b996c97942 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:40.273413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 675/1363] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:42.782520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae82a39-60f9-4f11-b46f-5d97ddbb2401 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:42.783135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae82a39-60f9-4f11-b46f-5d97ddbb2401 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:42.783135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae82a39-60f9-4f11-b46f-5d97ddbb2401 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:42.784509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 689/1364] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:42.789778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54da306-fde4-4d9e-9642-b7b6c4950555 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:42.790180 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54da306-fde4-4d9e-9642-b7b6c4950555 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:42.790451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54da306-fde4-4d9e-9642-b7b6c4950555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:42.809235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c54da306-fde4-4d9e-9642-b7b6c4950555 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:42.810927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c54da306-fde4-4d9e-9642-b7b6c4950555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:42.820366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 676/1365] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:42.865094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2107-80a3-4972-b7c9-af5c92a611bb None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:42.865386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2107-80a3-4972-b7c9-af5c92a611bb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:42.865664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2107-80a3-4972-b7c9-af5c92a611bb None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:42.867068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 690/1366] 10.222.0.62 () {54 vars in 961 bytes} [Wed Jun 15 15:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:42.872602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7205ee-10ad-4945-ae71-9365a6272ae5 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:42.872903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7205ee-10ad-4945-ae71-9365a6272ae5 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:42.873184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7205ee-10ad-4945-ae71-9365a6272ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:42.891976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0c7205ee-10ad-4945-ae71-9365a6272ae5 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:42.893202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0c7205ee-10ad-4945-ae71-9365a6272ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:42.908542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 677/1367] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Jun 15 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:42:42.916724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2107-80a3-4972-b7c9-af5c92a611bb None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:42.931918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2107-80a3-4972-b7c9-af5c92a611bb None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:42.937328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885abc40-f3cd-4ca2-898a-fa2787eab43c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:42.937962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885abc40-f3cd-4ca2-898a-fa2787eab43c service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:42.938247 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885abc40-f3cd-4ca2-898a-fa2787eab43c service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:42.938527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885abc40-f3cd-4ca2-898a-fa2787eab43c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:42.948217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885abc40-f3cd-4ca2-898a-fa2787eab43c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:42.950041 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885abc40-f3cd-4ca2-898a-fa2787eab43c service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:42.959857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 691/1368] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:45.171241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454229d-5d23-4f60-8b07-1da2972e917b None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:45.172975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454229d-5d23-4f60-8b07-1da2972e917b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:45.173475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454229d-5d23-4f60-8b07-1da2972e917b None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:45.180914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be4552-0589-4513-9714-e17b1753083a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:45.181896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be4552-0589-4513-9714-e17b1753083a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:45.182383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be4552-0589-4513-9714-e17b1753083a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:45.184454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 678/1369] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:45.202363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-c4be4552-0589-4513-9714-e17b1753083a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:45.204155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c4be4552-0589-4513-9714-e17b1753083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:45.221934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 692/1370] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:45.262525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454229d-5d23-4f60-8b07-1da2972e917b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:45.283085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454229d-5d23-4f60-8b07-1da2972e917b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:45.288836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80f433-b0b8-479b-bb84-852547dbff22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:45.289485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80f433-b0b8-479b-bb84-852547dbff22 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:45.289765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80f433-b0b8-479b-bb84-852547dbff22 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:45.290039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80f433-b0b8-479b-bb84-852547dbff22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:45.300413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80f433-b0b8-479b-bb84-852547dbff22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:45.302204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80f433-b0b8-479b-bb84-852547dbff22 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:45.312553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 679/1371] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:45.685067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2c820-d0f7-41ea-ac76-b7f156be7fbd None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:45.685358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2c820-d0f7-41ea-ac76-b7f156be7fbd None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:45.685928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2c820-d0f7-41ea-ac76-b7f156be7fbd None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:45.687952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 693/1372] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:45.693201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d459b4-e93d-41f7-949c-2cf53b4249db None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:45.693497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d459b4-e93d-41f7-949c-2cf53b4249db None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:45.693776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d459b4-e93d-41f7-949c-2cf53b4249db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:45.712484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-63d459b4-e93d-41f7-949c-2cf53b4249db None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:45.713968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-63d459b4-e93d-41f7-949c-2cf53b4249db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:45.722899 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 680/1373] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:46.027248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb3949-922e-4c3b-85db-4b2e3f37b70d None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:46.027248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb3949-922e-4c3b-85db-4b2e3f37b70d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:46.027248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb3949-922e-4c3b-85db-4b2e3f37b70d None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:46.029270 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 694/1374] 10.222.0.62 () {54 vars in 963 bytes} [Wed Jun 15 15:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:46.033146 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9ceea-9b8f-40fd-8bbe-98c7d7f7ff51 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:46.033442 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9ceea-9b8f-40fd-8bbe-98c7d7f7ff51 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:46.033721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9ceea-9b8f-40fd-8bbe-98c7d7f7ff51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:46.041248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:42:46.085284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-a4b9ceea-9b8f-40fd-8bbe-98c7d7f7ff51 None None] MFA Rules not processed for user `1bd66c1b7c7147a281ef2e7253224069`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:46.086470 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a4b9ceea-9b8f-40fd-8bbe-98c7d7f7ff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:46.149007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 681/1375] 10.222.0.62 () {58 vars in 1060 bytes} [Wed Jun 15 15:42:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:46.156397 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb3949-922e-4c3b-85db-4b2e3f37b70d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:46.166931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb3949-922e-4c3b-85db-4b2e3f37b70d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:46.170411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29049a3-4d5c-48b2-aded-1b29f633b68f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:46.171084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29049a3-4d5c-48b2-aded-1b29f633b68f service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:46.171937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29049a3-4d5c-48b2-aded-1b29f633b68f service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:46.172218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29049a3-4d5c-48b2-aded-1b29f633b68f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:46.181326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29049a3-4d5c-48b2-aded-1b29f633b68f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:46.182767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29049a3-4d5c-48b2-aded-1b29f633b68f service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:46.193179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 695/1376] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:47.492399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eacca-789f-4a7e-a4ea-6e36daf3dd8a None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:47.492987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eacca-789f-4a7e-a4ea-6e36daf3dd8a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:47.492987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eacca-789f-4a7e-a4ea-6e36daf3dd8a None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:47.494406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 682/1377] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:47.499721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e413140-debe-4d60-9a7e-b783557cc60b None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:47.500137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e413140-debe-4d60-9a7e-b783557cc60b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:47.500429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e413140-debe-4d60-9a7e-b783557cc60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:47.518911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6e413140-debe-4d60-9a7e-b783557cc60b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:47.520122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6e413140-debe-4d60-9a7e-b783557cc60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:47.527949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 696/1378] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:42:47.572665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98eacca-789f-4a7e-a4ea-6e36daf3dd8a None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:47.587512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98eacca-789f-4a7e-a4ea-6e36daf3dd8a None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:47.592937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7f9e5-4be6-4cd1-9fbf-43d41cc4cb1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:47.594121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7f9e5-4be6-4cd1-9fbf-43d41cc4cb1c service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:47.594409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7f9e5-4be6-4cd1-9fbf-43d41cc4cb1c service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:47.594709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7f9e5-4be6-4cd1-9fbf-43d41cc4cb1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:47.603037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7f9e5-4be6-4cd1-9fbf-43d41cc4cb1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:47.604655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7f9e5-4be6-4cd1-9fbf-43d41cc4cb1c service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:47.614453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 683/1379] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:49.802015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6578065-49a7-4f2f-bbed-2ce96cc87884 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:49.804129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6578065-49a7-4f2f-bbed-2ce96cc87884 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:49.804566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6578065-49a7-4f2f-bbed-2ce96cc87884 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:49.806498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 697/1380] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:49.811411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314a84d-faf1-4d70-8dd2-a215d5a085a7 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:49.811863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314a84d-faf1-4d70-8dd2-a215d5a085a7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:49.812197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314a84d-faf1-4d70-8dd2-a215d5a085a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:49.830330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-3314a84d-faf1-4d70-8dd2-a215d5a085a7 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:49.831674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3314a84d-faf1-4d70-8dd2-a215d5a085a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:49.839761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 684/1381] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:49.884800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6578065-49a7-4f2f-bbed-2ce96cc87884 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:49.899675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6578065-49a7-4f2f-bbed-2ce96cc87884 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:49.905247 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdb2be-7e56-4d88-bdbd-aa437ee6516b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:49.906529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdb2be-7e56-4d88-bdbd-aa437ee6516b service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:49.906845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdb2be-7e56-4d88-bdbd-aa437ee6516b service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:49.907127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdb2be-7e56-4d88-bdbd-aa437ee6516b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:49.915983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdb2be-7e56-4d88-bdbd-aa437ee6516b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:49.917570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdb2be-7e56-4d88-bdbd-aa437ee6516b service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:49.928011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 698/1382] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:52.101918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f2a46-74cc-422f-8bdc-10fc061d35b9 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:52.102426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f2a46-74cc-422f-8bdc-10fc061d35b9 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:52.102509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f2a46-74cc-422f-8bdc-10fc061d35b9 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:52.103866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 685/1383] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:52.109180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fb9a7-8b24-4694-9a14-f1c12dfc3c7a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:52.109615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fb9a7-8b24-4694-9a14-f1c12dfc3c7a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:52.109729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fb9a7-8b24-4694-9a14-f1c12dfc3c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:52.127433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4d2fb9a7-8b24-4694-9a14-f1c12dfc3c7a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:52.128623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4d2fb9a7-8b24-4694-9a14-f1c12dfc3c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:52.137368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 699/1384] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:52.179417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f2a46-74cc-422f-8bdc-10fc061d35b9 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:52.194696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f2a46-74cc-422f-8bdc-10fc061d35b9 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:52.200413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a9c94-b976-4856-93ba-99dc1b962d79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:52.201167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9c94-b976-4856-93ba-99dc1b962d79 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:52.201557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9c94-b976-4856-93ba-99dc1b962d79 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:52.201971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9c94-b976-4856-93ba-99dc1b962d79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:52.211236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a9c94-b976-4856-93ba-99dc1b962d79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:52.212925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a9c94-b976-4856-93ba-99dc1b962d79 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:52.223533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 686/1385] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:42:54.385340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4224bc-5f05-44b4-adaa-14a54b146ef6 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:54.385952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4224bc-5f05-44b4-adaa-14a54b146ef6 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:54.385952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4224bc-5f05-44b4-adaa-14a54b146ef6 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:54.387325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 700/1386] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:54.393699 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a911-268c-4b61-93a6-8725f7e97f73 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:54.394149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a911-268c-4b61-93a6-8725f7e97f73 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:54.394446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a911-268c-4b61-93a6-8725f7e97f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:54.413793 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-2058a911-268c-4b61-93a6-8725f7e97f73 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:54.415030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2058a911-268c-4b61-93a6-8725f7e97f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:54.423013 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 687/1387] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:54.467393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4224bc-5f05-44b4-adaa-14a54b146ef6 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:54.484105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4224bc-5f05-44b4-adaa-14a54b146ef6 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:54.489622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4521a8-20a7-44f7-b779-d18b5a930611 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:54.490278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4521a8-20a7-44f7-b779-d18b5a930611 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:54.490561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4521a8-20a7-44f7-b779-d18b5a930611 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:54.490869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4521a8-20a7-44f7-b779-d18b5a930611 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:54.499898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4521a8-20a7-44f7-b779-d18b5a930611 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:54.501484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4521a8-20a7-44f7-b779-d18b5a930611 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:54.512667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 701/1388] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:42:56.717002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a05c5-f180-4406-9172-99d62fc7b06b None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:56.717569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a05c5-f180-4406-9172-99d62fc7b06b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:56.718170 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a05c5-f180-4406-9172-99d62fc7b06b None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:56.720112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 688/1389] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:56.725462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3862-42e0-4c49-bc3c-68566c106990 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:56.726008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3862-42e0-4c49-bc3c-68566c106990 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:56.726008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3862-42e0-4c49-bc3c-68566c106990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:56.743958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-bf2f3862-42e0-4c49-bc3c-68566c106990 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:56.745215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bf2f3862-42e0-4c49-bc3c-68566c106990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:56.753390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 702/1390] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:56.799566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a05c5-f180-4406-9172-99d62fc7b06b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:56.813945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a05c5-f180-4406-9172-99d62fc7b06b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:56.819818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b066-c556-4b34-8957-40429383aebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:56.820451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b066-c556-4b34-8957-40429383aebc service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:56.820733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b066-c556-4b34-8957-40429383aebc service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:56.821009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b066-c556-4b34-8957-40429383aebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:56.830577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5b066-c556-4b34-8957-40429383aebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:56.832230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5b066-c556-4b34-8957-40429383aebc service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:56.841680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 689/1391] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:42:59.028199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764c9d4-3b1a-44d9-ab0d-bcc7b0ff4a24 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:59.028853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764c9d4-3b1a-44d9-ab0d-bcc7b0ff4a24 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:59.028853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764c9d4-3b1a-44d9-ab0d-bcc7b0ff4a24 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:59.030122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 703/1392] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:42:59.035276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff57b1-c24c-43a0-aaf6-b3067ddc344a None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:59.035756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff57b1-c24c-43a0-aaf6-b3067ddc344a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:59.035901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff57b1-c24c-43a0-aaf6-b3067ddc344a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:59.054181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-2dff57b1-c24c-43a0-aaf6-b3067ddc344a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:42:59.055421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2dff57b1-c24c-43a0-aaf6-b3067ddc344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:42:59.063874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 690/1393] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:42:59.111521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764c9d4-3b1a-44d9-ab0d-bcc7b0ff4a24 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:42:59.127664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764c9d4-3b1a-44d9-ab0d-bcc7b0ff4a24 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:42:59.133871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88155222-4a0d-47fd-bfb0-1813f95746a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:42:59.134546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88155222-4a0d-47fd-bfb0-1813f95746a6 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:42:59.134870 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88155222-4a0d-47fd-bfb0-1813f95746a6 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:42:59.135166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88155222-4a0d-47fd-bfb0-1813f95746a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:42:59.144112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88155222-4a0d-47fd-bfb0-1813f95746a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:42:59.145764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88155222-4a0d-47fd-bfb0-1813f95746a6 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:42:59.156422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 704/1394] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:01.318279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b06c4d-b873-43b7-93c1-e53adeabdf6c None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:01.318899 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b06c4d-b873-43b7-93c1-e53adeabdf6c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:01.318899 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b06c4d-b873-43b7-93c1-e53adeabdf6c None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:01.320237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 691/1395] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:01.325556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9571e-39cd-458d-b202-d43f18f6f4d8 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:01.325984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9571e-39cd-458d-b202-d43f18f6f4d8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:01.326129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9571e-39cd-458d-b202-d43f18f6f4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:01.345146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b8b9571e-39cd-458d-b202-d43f18f6f4d8 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:01.346393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b8b9571e-39cd-458d-b202-d43f18f6f4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:01.354646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 705/1396] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:01.399385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b06c4d-b873-43b7-93c1-e53adeabdf6c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:01.414125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b06c4d-b873-43b7-93c1-e53adeabdf6c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:01.416336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91030a-37bd-4b66-9b06-b4b8dff5081d None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:01.416627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91030a-37bd-4b66-9b06-b4b8dff5081d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:01.416912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91030a-37bd-4b66-9b06-b4b8dff5081d None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:01.418273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 706/1397] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:43:01.419811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5748563-8826-4ed8-af73-1669f4452b9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:01.420453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5748563-8826-4ed8-af73-1669f4452b9c service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:01.420740 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5748563-8826-4ed8-af73-1669f4452b9c service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:01.421022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5748563-8826-4ed8-af73-1669f4452b9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:01.423281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed179736-7331-4a54-a97c-7833a054da58 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:01.423565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed179736-7331-4a54-a97c-7833a054da58 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:01.423843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed179736-7331-4a54-a97c-7833a054da58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:01.429425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5748563-8826-4ed8-af73-1669f4452b9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:01.431035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5748563-8826-4ed8-af73-1669f4452b9c service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:01.441736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 692/1398] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:01.442009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ed179736-7331-4a54-a97c-7833a054da58 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:01.443311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ed179736-7331-4a54-a97c-7833a054da58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:01.454127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 707/1399] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:43:01.515878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac0dad-0203-4ae8-9e12-ce4eff3bf400 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:01.516168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac0dad-0203-4ae8-9e12-ce4eff3bf400 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:01.516450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac0dad-0203-4ae8-9e12-ce4eff3bf400 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:01.518097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 693/1400] 10.222.0.62 () {54 vars in 962 bytes} [Wed Jun 15 15:43:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:01.524044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aa873-ca5c-490a-8790-e622513692c1 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:01.524339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aa873-ca5c-490a-8790-e622513692c1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:01.524638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2aa873-ca5c-490a-8790-e622513692c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:01.578996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-de2aa873-ca5c-490a-8790-e622513692c1 None None] MFA Rules not processed for user `5f7405c8bc8e4c84bb056cd9e1df61c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:01.580185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-de2aa873-ca5c-490a-8790-e622513692c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:01.647385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 708/1401] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Jun 15 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:01.655365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac0dad-0203-4ae8-9e12-ce4eff3bf400 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:01.666845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac0dad-0203-4ae8-9e12-ce4eff3bf400 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:01.670342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfe4d9-f6ab-4e8b-9c9c-25e04068748a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:01.671512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe4d9-f6ab-4e8b-9c9c-25e04068748a service glance] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:01.671803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe4d9-f6ab-4e8b-9c9c-25e04068748a service glance] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:01.672080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe4d9-f6ab-4e8b-9c9c-25e04068748a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:01.681186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfe4d9-f6ab-4e8b-9c9c-25e04068748a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:01.682603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfe4d9-f6ab-4e8b-9c9c-25e04068748a service glance] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:01.692106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 694/1402] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:02.402422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc926d-da6c-42d4-b31d-c65657772b4a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:02.402981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc926d-da6c-42d4-b31d-c65657772b4a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:02.403123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc926d-da6c-42d4-b31d-c65657772b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:02.420868 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:43:02.454018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-fcfc926d-da6c-42d4-b31d-c65657772b4a None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:02.455320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fcfc926d-da6c-42d4-b31d-c65657772b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:02.521943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 709/1403] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:43:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:02.571027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f14dad-0a4f-4331-a054-0ef6b2975395 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:02.571586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f14dad-0a4f-4331-a054-0ef6b2975395 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:02.571722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f14dad-0a4f-4331-a054-0ef6b2975395 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:02.573223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 695/1404] 10.222.0.62 () {54 vars in 942 bytes} [Wed Jun 15 15:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:02.580474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cc467-5c9b-48c4-b4d8-1377c8a7ccbd None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:02.580997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cc467-5c9b-48c4-b4d8-1377c8a7ccbd None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:02.581280 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cc467-5c9b-48c4-b4d8-1377c8a7ccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:02.630421 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-dd4cc467-5c9b-48c4-b4d8-1377c8a7ccbd None None] MFA Rules not processed for user `a6e3bb6553264387936e9a534afc3ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:02.631928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-dd4cc467-5c9b-48c4-b4d8-1377c8a7ccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:02.702194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 710/1405] 10.222.0.62 () {58 vars in 1039 bytes} [Wed Jun 15 15:43:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:02.711570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f14dad-0a4f-4331-a054-0ef6b2975395 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:02.725046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f14dad-0a4f-4331-a054-0ef6b2975395 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:02.731022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1db26b-15a7-4435-ae63-371a030e719f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:02.731665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1db26b-15a7-4435-ae63-371a030e719f service swift] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:02.731950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1db26b-15a7-4435-ae63-371a030e719f service swift] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:02.732230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1db26b-15a7-4435-ae63-371a030e719f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:02.740742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1db26b-15a7-4435-ae63-371a030e719f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:02.742314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1db26b-15a7-4435-ae63-371a030e719f service swift] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:02.750987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 696/1406] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:43:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:43:03.618122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7560fe-e1d1-4fcd-8d8b-338f9cb80913 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:03.618678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7560fe-e1d1-4fcd-8d8b-338f9cb80913 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:03.618678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7560fe-e1d1-4fcd-8d8b-338f9cb80913 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:03.620138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 711/1407] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:03.625659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b306cd-4a92-4947-af36-1235dd522b49 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:03.626424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b306cd-4a92-4947-af36-1235dd522b49 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:03.626965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b306cd-4a92-4947-af36-1235dd522b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:03.648633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f7b306cd-4a92-4947-af36-1235dd522b49 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:03.649897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f7b306cd-4a92-4947-af36-1235dd522b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:03.659165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 697/1408] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:03.703933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7560fe-e1d1-4fcd-8d8b-338f9cb80913 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:03.718707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7560fe-e1d1-4fcd-8d8b-338f9cb80913 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:03.726900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38461ea7-a975-4b33-aebf-df3d3dd3cb91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:03.727625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38461ea7-a975-4b33-aebf-df3d3dd3cb91 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:03.727924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38461ea7-a975-4b33-aebf-df3d3dd3cb91 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:03.728182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38461ea7-a975-4b33-aebf-df3d3dd3cb91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:03.737241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38461ea7-a975-4b33-aebf-df3d3dd3cb91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:03.738854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38461ea7-a975-4b33-aebf-df3d3dd3cb91 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:03.749405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 712/1409] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:13.643658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=84 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:43:13.745819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=85 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:43:25.129362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c8cd8-77b7-484d-8938-32f314086b2f None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:25.129990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c8cd8-77b7-484d-8938-32f314086b2f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:25.129990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c8cd8-77b7-484d-8938-32f314086b2f None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:25.131322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 698/1410] 10.222.0.62 () {54 vars in 918 bytes} [Wed Jun 15 15:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:25.136678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f5d86-91be-463d-9568-8aa1c0ee7ffb None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:25.137136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f5d86-91be-463d-9568-8aa1c0ee7ffb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:25.137243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f5d86-91be-463d-9568-8aa1c0ee7ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:25.155233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-307f5d86-91be-463d-9568-8aa1c0ee7ffb None None] MFA Rules not processed for user `a786a95c48c348dbba269e0e33265271`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:25.156989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-307f5d86-91be-463d-9568-8aa1c0ee7ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:25.165310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 713/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Jun 15 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:25.179386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3210b-1705-4060-a2db-40c8589e686b None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:25.179677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3210b-1705-4060-a2db-40c8589e686b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:25.179955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3210b-1705-4060-a2db-40c8589e686b None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:25.181301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 699/1412] 10.222.0.62 () {54 vars in 918 bytes} [Wed Jun 15 15:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:25.185805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004e24-41b4-426e-a86e-946e0f087476 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:25.186093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004e24-41b4-426e-a86e-946e0f087476 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:25.186369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004e24-41b4-426e-a86e-946e0f087476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:25.204514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-13004e24-41b4-426e-a86e-946e0f087476 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:25.205688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-13004e24-41b4-426e-a86e-946e0f087476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:25.214554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 714/1413] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Jun 15 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:27.801731 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f97f6-eaee-4c38-93fd-6207872e1e35 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:27.802331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f97f6-eaee-4c38-93fd-6207872e1e35 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:27.802331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f97f6-eaee-4c38-93fd-6207872e1e35 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:27.803676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:27.808930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c669071-c656-4261-abc0-51637c221d21 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:27.809348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c669071-c656-4261-abc0-51637c221d21 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:27.809517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c669071-c656-4261-abc0-51637c221d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:27.827189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5c669071-c656-4261-abc0-51637c221d21 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:27.828491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5c669071-c656-4261-abc0-51637c221d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:27.836559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:27.881168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f97f6-eaee-4c38-93fd-6207872e1e35 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:27.896325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f97f6-eaee-4c38-93fd-6207872e1e35 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:27.901604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129fbe6-05d7-40d8-bdb8-ecbad96a2d1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:27.902240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129fbe6-05d7-40d8-bdb8-ecbad96a2d1c service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:27.902528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129fbe6-05d7-40d8-bdb8-ecbad96a2d1c service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:27.902839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129fbe6-05d7-40d8-bdb8-ecbad96a2d1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:27.911582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129fbe6-05d7-40d8-bdb8-ecbad96a2d1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:27.913291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129fbe6-05d7-40d8-bdb8-ecbad96a2d1c service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:27.923539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 701/1416] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:28.861739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902ff53-dc8a-472a-97c9-139901b4e286 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:28.862193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902ff53-dc8a-472a-97c9-139901b4e286 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:28.862288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902ff53-dc8a-472a-97c9-139901b4e286 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:28.863665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:28.868554 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd0286-0516-47c5-bac2-8777a5508ff6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:28.868970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd0286-0516-47c5-bac2-8777a5508ff6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:28.869112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd0286-0516-47c5-bac2-8777a5508ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:28.887707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f9dd0286-0516-47c5-bac2-8777a5508ff6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:28.888881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f9dd0286-0516-47c5-bac2-8777a5508ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:28.896689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:29.057583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902ff53-dc8a-472a-97c9-139901b4e286 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:29.072332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902ff53-dc8a-472a-97c9-139901b4e286 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:29.077723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356e262-116c-449f-a220-445b5c0c06ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:29.078356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356e262-116c-449f-a220-445b5c0c06ce service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:29.078640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356e262-116c-449f-a220-445b5c0c06ce service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:29.078954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356e262-116c-449f-a220-445b5c0c06ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:29.087019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356e262-116c-449f-a220-445b5c0c06ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:29.088680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356e262-116c-449f-a220-445b5c0c06ce service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:29.098471 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 717/1419] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:30.012169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0df9a0-814e-499f-acc3-3824f632bd99 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:30.012708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0df9a0-814e-499f-acc3-3824f632bd99 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:30.012805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0df9a0-814e-499f-acc3-3824f632bd99 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:30.014132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:30.019725 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994a756-1a98-49f8-a9c9-462cb0991b81 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:30.020118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994a756-1a98-49f8-a9c9-462cb0991b81 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:30.020277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994a756-1a98-49f8-a9c9-462cb0991b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:30.037771 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-a994a756-1a98-49f8-a9c9-462cb0991b81 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:30.039000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a994a756-1a98-49f8-a9c9-462cb0991b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:30.046597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:30.202289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0df9a0-814e-499f-acc3-3824f632bd99 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:30.217928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0df9a0-814e-499f-acc3-3824f632bd99 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:30.223924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef8fe5-b7e3-406b-85f6-8f2e28fd5e41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:30.224551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef8fe5-b7e3-406b-85f6-8f2e28fd5e41 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:30.224834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef8fe5-b7e3-406b-85f6-8f2e28fd5e41 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:30.225112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef8fe5-b7e3-406b-85f6-8f2e28fd5e41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:30.233287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef8fe5-b7e3-406b-85f6-8f2e28fd5e41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:30.234886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef8fe5-b7e3-406b-85f6-8f2e28fd5e41 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:30.244045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 704/1422] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:31.151516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733fe4-2335-4312-9931-5dfde3865345 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:31.152060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733fe4-2335-4312-9931-5dfde3865345 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:31.152060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733fe4-2335-4312-9931-5dfde3865345 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:31.153443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:31.158414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442036-a7c7-4b6b-943a-26fece678b7c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:31.158852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442036-a7c7-4b6b-943a-26fece678b7c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:31.161475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442036-a7c7-4b6b-943a-26fece678b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:31.180203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-89442036-a7c7-4b6b-943a-26fece678b7c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:31.181390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-89442036-a7c7-4b6b-943a-26fece678b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:31.191713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:43:31.340012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6733fe4-2335-4312-9931-5dfde3865345 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:31.354132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6733fe4-2335-4312-9931-5dfde3865345 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:31.359525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522f40c-70b9-4541-a0dc-1e036c2ac48b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:31.360660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522f40c-70b9-4541-a0dc-1e036c2ac48b service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:31.360939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522f40c-70b9-4541-a0dc-1e036c2ac48b service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:31.361215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522f40c-70b9-4541-a0dc-1e036c2ac48b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:31.369402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522f40c-70b9-4541-a0dc-1e036c2ac48b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:31.371012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522f40c-70b9-4541-a0dc-1e036c2ac48b service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:31.380731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 720/1425] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:32.295998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242bde1-0ab4-4900-a45f-582555582499 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:32.296563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242bde1-0ab4-4900-a45f-582555582499 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:32.296563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242bde1-0ab4-4900-a45f-582555582499 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:32.297932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:32.302824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb2c11-f335-4eb4-8eca-9f0a307b8902 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:32.303237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb2c11-f335-4eb4-8eca-9f0a307b8902 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:32.303388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb2c11-f335-4eb4-8eca-9f0a307b8902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:32.321476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-40eb2c11-f335-4eb4-8eca-9f0a307b8902 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:32.322672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-40eb2c11-f335-4eb4-8eca-9f0a307b8902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:32.331000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:32.478223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242bde1-0ab4-4900-a45f-582555582499 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:32.492616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242bde1-0ab4-4900-a45f-582555582499 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:32.497924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e224e-3897-46d0-9b77-3c0ff73bd0cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:32.498556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e224e-3897-46d0-9b77-3c0ff73bd0cb service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:32.498871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e224e-3897-46d0-9b77-3c0ff73bd0cb service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:32.499152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e224e-3897-46d0-9b77-3c0ff73bd0cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:32.507675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e224e-3897-46d0-9b77-3c0ff73bd0cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:32.509486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e224e-3897-46d0-9b77-3c0ff73bd0cb service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:32.519407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 707/1428] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:33.376143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923334bd-5edd-489f-97c4-2fd3f40b3f21 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:33.376781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923334bd-5edd-489f-97c4-2fd3f40b3f21 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:33.376781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923334bd-5edd-489f-97c4-2fd3f40b3f21 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:33.378059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:33.383019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2020f7-3a4a-4c69-b960-94c6ad7c2943 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:33.383399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2020f7-3a4a-4c69-b960-94c6ad7c2943 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:33.383610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2020f7-3a4a-4c69-b960-94c6ad7c2943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:33.401788 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-6b2020f7-3a4a-4c69-b960-94c6ad7c2943 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:33.403001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6b2020f7-3a4a-4c69-b960-94c6ad7c2943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:33.411301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:33.576447 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923334bd-5edd-489f-97c4-2fd3f40b3f21 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:33.590857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923334bd-5edd-489f-97c4-2fd3f40b3f21 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:33.596758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8089d61-0e7a-415d-8243-5d4b7281ad60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:33.597387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8089d61-0e7a-415d-8243-5d4b7281ad60 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:33.597665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8089d61-0e7a-415d-8243-5d4b7281ad60 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:33.597939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8089d61-0e7a-415d-8243-5d4b7281ad60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:33.605839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8089d61-0e7a-415d-8243-5d4b7281ad60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:33.607449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8089d61-0e7a-415d-8243-5d4b7281ad60 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:33.617039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 723/1431] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:34.460213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e6a8e-495d-4654-8655-e926ace2dbaf None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:34.460670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e6a8e-495d-4654-8655-e926ace2dbaf None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:34.460767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e6a8e-495d-4654-8655-e926ace2dbaf None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:34.462150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:34.467179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6997690-5050-4773-ac99-daa677d43c0d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:34.467595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6997690-5050-4773-ac99-daa677d43c0d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:34.467736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6997690-5050-4773-ac99-daa677d43c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:34.487212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b6997690-5050-4773-ac99-daa677d43c0d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:34.488403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b6997690-5050-4773-ac99-daa677d43c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:34.496122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:34.644474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e6a8e-495d-4654-8655-e926ace2dbaf None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:34.659208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e6a8e-495d-4654-8655-e926ace2dbaf None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:34.664595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd875380-463a-4893-bf7e-7736b00c8ab6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:34.665228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875380-463a-4893-bf7e-7736b00c8ab6 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:34.665513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875380-463a-4893-bf7e-7736b00c8ab6 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:34.665794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875380-463a-4893-bf7e-7736b00c8ab6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:34.674411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd875380-463a-4893-bf7e-7736b00c8ab6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:34.676020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd875380-463a-4893-bf7e-7736b00c8ab6 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:34.685226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 710/1434] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:35.523102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88960f-1c34-4a0b-b257-49b125614ded None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:35.523623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88960f-1c34-4a0b-b257-49b125614ded None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:35.523719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88960f-1c34-4a0b-b257-49b125614ded None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:35.525013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:35.530104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715849e-6219-40b3-b38f-1347d871c380 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:35.530506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715849e-6219-40b3-b38f-1347d871c380 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:35.530675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715849e-6219-40b3-b38f-1347d871c380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:35.548633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-2715849e-6219-40b3-b38f-1347d871c380 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:35.549826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2715849e-6219-40b3-b38f-1347d871c380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:35.557487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:35.707975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88960f-1c34-4a0b-b257-49b125614ded None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:35.721923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88960f-1c34-4a0b-b257-49b125614ded None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:35.727294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14508dd-9f0f-40c7-8bbc-2703e80d3546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:35.727922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14508dd-9f0f-40c7-8bbc-2703e80d3546 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:35.728204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14508dd-9f0f-40c7-8bbc-2703e80d3546 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:35.728498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14508dd-9f0f-40c7-8bbc-2703e80d3546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:35.737092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14508dd-9f0f-40c7-8bbc-2703e80d3546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:35.738656 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14508dd-9f0f-40c7-8bbc-2703e80d3546 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:35.747949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 726/1437] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:36.582203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cf794-d633-4f74-906c-480a59836707 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:36.582776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cf794-d633-4f74-906c-480a59836707 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:36.582776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cf794-d633-4f74-906c-480a59836707 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:36.584157 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:36.589021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c647d7d-ab50-4d42-80b6-a50d059192f8 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:36.589439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c647d7d-ab50-4d42-80b6-a50d059192f8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:36.589580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c647d7d-ab50-4d42-80b6-a50d059192f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:36.607798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-8c647d7d-ab50-4d42-80b6-a50d059192f8 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:36.608995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8c647d7d-ab50-4d42-80b6-a50d059192f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:36.616739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:36.765815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cf794-d633-4f74-906c-480a59836707 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:36.780388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cf794-d633-4f74-906c-480a59836707 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:36.785737 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f5a34-35c2-4382-8ffe-72bdc40cdeb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:36.786365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f5a34-35c2-4382-8ffe-72bdc40cdeb5 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:36.786649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f5a34-35c2-4382-8ffe-72bdc40cdeb5 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:36.786955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f5a34-35c2-4382-8ffe-72bdc40cdeb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:36.795057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f5a34-35c2-4382-8ffe-72bdc40cdeb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:36.796635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f5a34-35c2-4382-8ffe-72bdc40cdeb5 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:36.805906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 713/1440] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:38.812143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595b8a-4786-4bd4-bf14-7a98b06d99a5 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:38.812675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595b8a-4786-4bd4-bf14-7a98b06d99a5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:38.812675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595b8a-4786-4bd4-bf14-7a98b06d99a5 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:38.814663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:38.819784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33351484-75c3-417e-a8a6-e30e04725cf9 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:38.820228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33351484-75c3-417e-a8a6-e30e04725cf9 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:38.820361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33351484-75c3-417e-a8a6-e30e04725cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:38.837882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-33351484-75c3-417e-a8a6-e30e04725cf9 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:38.839086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-33351484-75c3-417e-a8a6-e30e04725cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:38.846609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:38.885576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595b8a-4786-4bd4-bf14-7a98b06d99a5 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:38.896414 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595b8a-4786-4bd4-bf14-7a98b06d99a5 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:38.899936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0f7f1-d8a6-400d-a30e-111d7a44bd51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:38.900566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0f7f1-d8a6-400d-a30e-111d7a44bd51 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:38.900847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0f7f1-d8a6-400d-a30e-111d7a44bd51 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:38.901122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0f7f1-d8a6-400d-a30e-111d7a44bd51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:38.910378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0f7f1-d8a6-400d-a30e-111d7a44bd51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:38.911840 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0f7f1-d8a6-400d-a30e-111d7a44bd51 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:38.921016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 729/1443] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:41.182970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f24fd-89af-4845-bf83-e7506f795db9 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:41.183570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f24fd-89af-4845-bf83-e7506f795db9 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:41.183570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f24fd-89af-4845-bf83-e7506f795db9 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:41.184911 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:41.190762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0f5d6-40c4-4006-a6d5-d2579754eada None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:41.191218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0f5d6-40c4-4006-a6d5-d2579754eada None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:41.191394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0f5d6-40c4-4006-a6d5-d2579754eada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:41.209176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5fa0f5d6-40c4-4006-a6d5-d2579754eada None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:41.210359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5fa0f5d6-40c4-4006-a6d5-d2579754eada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:41.218015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:41.256150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f24fd-89af-4845-bf83-e7506f795db9 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:41.269586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f24fd-89af-4845-bf83-e7506f795db9 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:41.273180 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cd025-02aa-4ef7-8567-df18734c3ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:41.273878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cd025-02aa-4ef7-8567-df18734c3ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:41.274165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cd025-02aa-4ef7-8567-df18734c3ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:41.274445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cd025-02aa-4ef7-8567-df18734c3ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:41.283457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cd025-02aa-4ef7-8567-df18734c3ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:41.284869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cd025-02aa-4ef7-8567-df18734c3ce9 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:41.294268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 716/1446] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:43.774393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c5e26-72f1-4730-9580-37fb0ebb54ea None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:43.774990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c5e26-72f1-4730-9580-37fb0ebb54ea None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:43.774990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c5e26-72f1-4730-9580-37fb0ebb54ea None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:43.776250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:43.781359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cc379-08fb-4a79-9f6e-0190c61c9fe4 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:43.781758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cc379-08fb-4a79-9f6e-0190c61c9fe4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:43.781954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cc379-08fb-4a79-9f6e-0190c61c9fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:43.800060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-a40cc379-08fb-4a79-9f6e-0190c61c9fe4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:43.801242 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a40cc379-08fb-4a79-9f6e-0190c61c9fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:43.808887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:43.847622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c5e26-72f1-4730-9580-37fb0ebb54ea None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:43.858120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c5e26-72f1-4730-9580-37fb0ebb54ea None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:43.862147 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e7c87-0a63-46e9-b398-d2968f1ebe92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:43.862806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e7c87-0a63-46e9-b398-d2968f1ebe92 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:43.863091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e7c87-0a63-46e9-b398-d2968f1ebe92 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:43.863369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e7c87-0a63-46e9-b398-d2968f1ebe92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:43.872091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e7c87-0a63-46e9-b398-d2968f1ebe92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:43.873502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e7c87-0a63-46e9-b398-d2968f1ebe92 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:43.883337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 732/1449] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:46.179100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096ce6-f7a5-4282-b88f-740d1b8b7cef None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:46.179694 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096ce6-f7a5-4282-b88f-740d1b8b7cef None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:46.179694 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096ce6-f7a5-4282-b88f-740d1b8b7cef None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:46.181020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:46.186296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c3faf-6583-4c9c-916e-32a0780417d7 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:46.186754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c3faf-6583-4c9c-916e-32a0780417d7 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:46.186895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c3faf-6583-4c9c-916e-32a0780417d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:46.204257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-795c3faf-6583-4c9c-916e-32a0780417d7 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:46.205950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-795c3faf-6583-4c9c-916e-32a0780417d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:46.213641 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:46.251997 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7096ce6-f7a5-4282-b88f-740d1b8b7cef None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:46.262883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7096ce6-f7a5-4282-b88f-740d1b8b7cef None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:46.266355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948da1f-d16f-43c1-9ea2-f270c0fe272c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:46.267023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948da1f-d16f-43c1-9ea2-f270c0fe272c service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:46.267309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948da1f-d16f-43c1-9ea2-f270c0fe272c service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:46.267589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948da1f-d16f-43c1-9ea2-f270c0fe272c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:46.277086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948da1f-d16f-43c1-9ea2-f270c0fe272c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:46.278527 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948da1f-d16f-43c1-9ea2-f270c0fe272c service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:46.288767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 719/1452] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:48.590493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d3e09-8ebb-464b-9f34-8a7e27174b4c None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:48.591055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d3e09-8ebb-464b-9f34-8a7e27174b4c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:48.591055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d3e09-8ebb-464b-9f34-8a7e27174b4c None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:48.592669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:48.597715 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf2746-82b0-429e-80e6-cd0903dc38f4 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:48.598114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf2746-82b0-429e-80e6-cd0903dc38f4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:48.598279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf2746-82b0-429e-80e6-cd0903dc38f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:48.616379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-eecf2746-82b0-429e-80e6-cd0903dc38f4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:48.617569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-eecf2746-82b0-429e-80e6-cd0903dc38f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:48.625315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:48.664578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d3e09-8ebb-464b-9f34-8a7e27174b4c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:48.676211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d3e09-8ebb-464b-9f34-8a7e27174b4c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:48.679747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76e479-a72e-46bc-ab98-3e7c096d61f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:48.680382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76e479-a72e-46bc-ab98-3e7c096d61f8 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:48.680662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76e479-a72e-46bc-ab98-3e7c096d61f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:48.680938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76e479-a72e-46bc-ab98-3e7c096d61f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:48.689725 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76e479-a72e-46bc-ab98-3e7c096d61f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:48.691196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76e479-a72e-46bc-ab98-3e7c096d61f8 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:48.700483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 735/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:50.863870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50023b2-76e8-44b4-81b6-39ef4337e47e None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:50.864400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50023b2-76e8-44b4-81b6-39ef4337e47e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:50.864400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50023b2-76e8-44b4-81b6-39ef4337e47e None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:50.865948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:50.871152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39566293-f178-4a05-9da2-e0c06bf185f6 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:50.871594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39566293-f178-4a05-9da2-e0c06bf185f6 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:50.871703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39566293-f178-4a05-9da2-e0c06bf185f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:50.889910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-39566293-f178-4a05-9da2-e0c06bf185f6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:50.891138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-39566293-f178-4a05-9da2-e0c06bf185f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:50.898903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:50.937299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50023b2-76e8-44b4-81b6-39ef4337e47e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:50.947945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50023b2-76e8-44b4-81b6-39ef4337e47e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:50.951424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468b18b-273b-4805-8240-5bdc71a5b1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:50.952057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468b18b-273b-4805-8240-5bdc71a5b1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:50.952342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468b18b-273b-4805-8240-5bdc71a5b1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:50.952620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468b18b-273b-4805-8240-5bdc71a5b1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:50.961367 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468b18b-273b-4805-8240-5bdc71a5b1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:50.962818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468b18b-273b-4805-8240-5bdc71a5b1b8 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:50.972611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 722/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:53.141801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5677e4c-5dff-4140-b47a-45430a408406 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:53.142352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5677e4c-5dff-4140-b47a-45430a408406 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:53.142352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5677e4c-5dff-4140-b47a-45430a408406 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:53.144304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:53.149672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c9c41-49fd-4962-b2f3-8cd31e7265db None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:53.150071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c9c41-49fd-4962-b2f3-8cd31e7265db None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:53.150226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c9c41-49fd-4962-b2f3-8cd31e7265db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:53.168081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-497c9c41-49fd-4962-b2f3-8cd31e7265db None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:53.169247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-497c9c41-49fd-4962-b2f3-8cd31e7265db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:53.176913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:53.214794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5677e4c-5dff-4140-b47a-45430a408406 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:53.225331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5677e4c-5dff-4140-b47a-45430a408406 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:53.228838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648de8c-db43-4d1c-8feb-93ad820ba230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:53.229639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648de8c-db43-4d1c-8feb-93ad820ba230 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:53.229782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648de8c-db43-4d1c-8feb-93ad820ba230 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:53.230175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648de8c-db43-4d1c-8feb-93ad820ba230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:53.239106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648de8c-db43-4d1c-8feb-93ad820ba230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:53.240529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648de8c-db43-4d1c-8feb-93ad820ba230 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:53.249768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 738/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:55.422168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810c21-a684-45f9-8631-9d0dc4431211 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:55.422679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810c21-a684-45f9-8631-9d0dc4431211 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:55.422679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810c21-a684-45f9-8631-9d0dc4431211 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:55.424075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:55.429964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7459bf-4a1c-4d46-9cc2-9f54082a10ee None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:55.430399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7459bf-4a1c-4d46-9cc2-9f54082a10ee None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:55.430522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7459bf-4a1c-4d46-9cc2-9f54082a10ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:55.436599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=79 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:43:55.449187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9d7459bf-4a1c-4d46-9cc2-9f54082a10ee None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:55.450378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9d7459bf-4a1c-4d46-9cc2-9f54082a10ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:55.458169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:55.495833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92810c21-a684-45f9-8631-9d0dc4431211 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:55.501123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=73 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:43:55.507441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92810c21-a684-45f9-8631-9d0dc4431211 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:55.510871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05c46e-8094-4d20-8d12-8f0724efe20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:55.511498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05c46e-8094-4d20-8d12-8f0724efe20a service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:55.511778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05c46e-8094-4d20-8d12-8f0724efe20a service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:55.512055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05c46e-8094-4d20-8d12-8f0724efe20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:55.521005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05c46e-8094-4d20-8d12-8f0724efe20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:55.522408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05c46e-8094-4d20-8d12-8f0724efe20a service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:55.531523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 725/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:43:57.672098 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e0e2-49e0-47a4-ab53-bee2c03dd3d8 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:57.672656 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e0e2-49e0-47a4-ab53-bee2c03dd3d8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:57.672656 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e0e2-49e0-47a4-ab53-bee2c03dd3d8 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:57.673995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:57.679235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de8333-c145-4e2e-95b9-6ffcd6519571 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:57.679636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de8333-c145-4e2e-95b9-6ffcd6519571 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:57.679809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de8333-c145-4e2e-95b9-6ffcd6519571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:57.697618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-a8de8333-c145-4e2e-95b9-6ffcd6519571 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:57.698923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a8de8333-c145-4e2e-95b9-6ffcd6519571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:57.706807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:57.742990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e0e2-49e0-47a4-ab53-bee2c03dd3d8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:57.754104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e0e2-49e0-47a4-ab53-bee2c03dd3d8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:57.757617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e57238-6dde-4bac-ac72-15a2f5fef798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:57.758256 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e57238-6dde-4bac-ac72-15a2f5fef798 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:57.758540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e57238-6dde-4bac-ac72-15a2f5fef798 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:57.758906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e57238-6dde-4bac-ac72-15a2f5fef798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:57.768287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e57238-6dde-4bac-ac72-15a2f5fef798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:43:57.770319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e57238-6dde-4bac-ac72-15a2f5fef798 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:43:57.779639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 741/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:43:59.909294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bee6d6-51b4-45a2-8af9-ca3ae996e154 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:59.909863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bee6d6-51b4-45a2-8af9-ca3ae996e154 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:59.909863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bee6d6-51b4-45a2-8af9-ca3ae996e154 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:59.911790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:43:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:43:59.916857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42724e7f-af80-49b4-83eb-b097b5aa97fc None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:59.917263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42724e7f-af80-49b4-83eb-b097b5aa97fc None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:59.917420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42724e7f-af80-49b4-83eb-b097b5aa97fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:43:59.934754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-42724e7f-af80-49b4-83eb-b097b5aa97fc None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:43:59.935972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-42724e7f-af80-49b4-83eb-b097b5aa97fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:43:59.944185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:43:59.980012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bee6d6-51b4-45a2-8af9-ca3ae996e154 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:43:59.990450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bee6d6-51b4-45a2-8af9-ca3ae996e154 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:43:59.993930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39b059-0650-4345-aaa4-e815412af4ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:43:59.994553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39b059-0650-4345-aaa4-e815412af4ec service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:43:59.994869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39b059-0650-4345-aaa4-e815412af4ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:43:59.995141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39b059-0650-4345-aaa4-e815412af4ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:00.004201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39b059-0650-4345-aaa4-e815412af4ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:44:00.005619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39b059-0650-4345-aaa4-e815412af4ec service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:44:00.015064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 728/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:43:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:44:09.929684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=11 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:44:10.010757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=14 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:44:19.913771 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f800-7bc8-453f-8ec3-7cc66f97f6e1 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:19.914388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f800-7bc8-453f-8ec3-7cc66f97f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:19.914388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f800-7bc8-453f-8ec3-7cc66f97f6e1 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:19.915704 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 743/1471] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:44:19.920839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9485c-eed5-4667-b221-16487da2b65f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:19.921276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9485c-eed5-4667-b221-16487da2b65f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:19.921411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9485c-eed5-4667-b221-16487da2b65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:19.939307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-58e9485c-eed5-4667-b221-16487da2b65f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:44:19.940477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-58e9485c-eed5-4667-b221-16487da2b65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:44:19.948031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 729/1472] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:44:19.994679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcc2a2-2729-4c3c-bf74-d22c54d02ae3 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:19.995003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcc2a2-2729-4c3c-bf74-d22c54d02ae3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:19.995281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcc2a2-2729-4c3c-bf74-d22c54d02ae3 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:19.996664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 744/1473] 10.222.0.62 () {54 vars in 962 bytes} [Wed Jun 15 15:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:44:20.001348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc8bd8-fd1e-4d03-ad38-b4b7a1fd4969 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:20.001640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc8bd8-fd1e-4d03-ad38-b4b7a1fd4969 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:20.001920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc8bd8-fd1e-4d03-ad38-b4b7a1fd4969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:20.019649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b7cc8bd8-fd1e-4d03-ad38-b4b7a1fd4969 None None] MFA Rules not processed for user `5f7405c8bc8e4c84bb056cd9e1df61c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:44:20.020821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b7cc8bd8-fd1e-4d03-ad38-b4b7a1fd4969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:44:20.027739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 730/1474] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Jun 15 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:44:20.034749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcc2a2-2729-4c3c-bf74-d22c54d02ae3 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:44:20.045621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcc2a2-2729-4c3c-bf74-d22c54d02ae3 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:44:20.049097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346db3dd-f7c2-468f-815b-6e0800fdf7e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:44:20.049717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346db3dd-f7c2-468f-815b-6e0800fdf7e6 service glance] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:20.050002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346db3dd-f7c2-468f-815b-6e0800fdf7e6 service glance] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:20.050270 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346db3dd-f7c2-468f-815b-6e0800fdf7e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:20.059059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346db3dd-f7c2-468f-815b-6e0800fdf7e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:44:20.060466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346db3dd-f7c2-468f-815b-6e0800fdf7e6 service glance] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:44:20.069591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 745/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:44:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:44:22.284869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a3c57-464f-4484-981b-2b75396ef766 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:22.285423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a3c57-464f-4484-981b-2b75396ef766 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:22.285423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a3c57-464f-4484-981b-2b75396ef766 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:22.286900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 731/1476] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:44:22.291980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c651d6-563a-4d8f-91f2-7ecb09d5da79 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:22.292403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c651d6-563a-4d8f-91f2-7ecb09d5da79 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:22.292576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c651d6-563a-4d8f-91f2-7ecb09d5da79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:22.310540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e6c651d6-563a-4d8f-91f2-7ecb09d5da79 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:44:22.311779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e6c651d6-563a-4d8f-91f2-7ecb09d5da79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:44:22.320147 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 746/1477] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:44:22.342843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a3c57-464f-4484-981b-2b75396ef766 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:44:22.355082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a3c57-464f-4484-981b-2b75396ef766 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:44:22.359418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80006845-643d-4938-8afc-b0c5a8197fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:44:22.360050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80006845-643d-4938-8afc-b0c5a8197fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:22.360329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80006845-643d-4938-8afc-b0c5a8197fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:22.360603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80006845-643d-4938-8afc-b0c5a8197fb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:22.361318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80006845-643d-4938-8afc-b0c5a8197fb4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:44:22.363100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING py.warnings [None req-80006845-643d-4938-8afc-b0c5a8197fb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:44:22.363100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: warnings.warn(msg) Jun 15 15:44:22.363100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: Jun 15 15:44:22.364227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80006845-643d-4938-8afc-b0c5a8197fb4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:44:22.383551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 732/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Jun 15 15:44:22 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:44:24.510007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107292a1-9583-40d7-9976-d9627bd248cf None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:24.510567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107292a1-9583-40d7-9976-d9627bd248cf None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:24.510567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107292a1-9583-40d7-9976-d9627bd248cf None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:24.511966 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 747/1479] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:44:24.517544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd43fac-8a3b-4fb8-b6d1-8d6ef2fcb25d None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:24.517938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd43fac-8a3b-4fb8-b6d1-8d6ef2fcb25d None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:24.518098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd43fac-8a3b-4fb8-b6d1-8d6ef2fcb25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:24.535376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-dcd43fac-8a3b-4fb8-b6d1-8d6ef2fcb25d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:44:24.536557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-dcd43fac-8a3b-4fb8-b6d1-8d6ef2fcb25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:44:24.544197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 733/1480] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:44:24.586172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107292a1-9583-40d7-9976-d9627bd248cf None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:44:24.600402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107292a1-9583-40d7-9976-d9627bd248cf None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:44:24.605831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212c534-83a3-4640-877e-ba426e21aa6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:44:24.606459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212c534-83a3-4640-877e-ba426e21aa6c service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:24.606753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212c534-83a3-4640-877e-ba426e21aa6c service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:24.607050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212c534-83a3-4640-877e-ba426e21aa6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:24.615213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212c534-83a3-4640-877e-ba426e21aa6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:44:24.617347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212c534-83a3-4640-877e-ba426e21aa6c service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:44:24.626698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 748/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:44:26.904554 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21d966-cbfc-4a73-9da3-8bc7c44c8814 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:26.904996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21d966-cbfc-4a73-9da3-8bc7c44c8814 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:26.905117 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21d966-cbfc-4a73-9da3-8bc7c44c8814 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:26.906493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 734/1482] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:44:26.911681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902147-aabe-49f9-a206-61a478b8b90f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:26.912083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902147-aabe-49f9-a206-61a478b8b90f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:26.912240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902147-aabe-49f9-a206-61a478b8b90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:26.930599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-d6902147-aabe-49f9-a206-61a478b8b90f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:44:26.931821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d6902147-aabe-49f9-a206-61a478b8b90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:44:26.939582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 749/1483] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:44:26.975692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21d966-cbfc-4a73-9da3-8bc7c44c8814 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:44:26.986214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21d966-cbfc-4a73-9da3-8bc7c44c8814 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:44:26.989662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c13bf2-9ebe-4a7e-aad5-ce7f5bd08af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:44:26.990292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c13bf2-9ebe-4a7e-aad5-ce7f5bd08af5 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:26.990593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c13bf2-9ebe-4a7e-aad5-ce7f5bd08af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:26.991413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c13bf2-9ebe-4a7e-aad5-ce7f5bd08af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:27.000173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c13bf2-9ebe-4a7e-aad5-ce7f5bd08af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:44:27.001585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c13bf2-9ebe-4a7e-aad5-ce7f5bd08af5 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:44:27.011464 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 735/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:44:29.182488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb0db7-8ab8-4ec9-b521-9735b85f8cab None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:29.183045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb0db7-8ab8-4ec9-b521-9735b85f8cab None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:29.183045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb0db7-8ab8-4ec9-b521-9735b85f8cab None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:29.184382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 750/1485] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:44:29.189427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b8709-083c-48a9-9b03-f2303fec5357 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:29.189802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b8709-083c-48a9-9b03-f2303fec5357 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:29.189995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b8709-083c-48a9-9b03-f2303fec5357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:29.207604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-464b8709-083c-48a9-9b03-f2303fec5357 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:44:29.208800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-464b8709-083c-48a9-9b03-f2303fec5357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:44:29.216553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 736/1486] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:44:29.255181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb0db7-8ab8-4ec9-b521-9735b85f8cab None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:44:29.265907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb0db7-8ab8-4ec9-b521-9735b85f8cab None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:44:29.269745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dfb7b-69e2-4baf-af03-9be2f556dc6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:44:29.270375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dfb7b-69e2-4baf-af03-9be2f556dc6f service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:29.270661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dfb7b-69e2-4baf-af03-9be2f556dc6f service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:29.270969 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dfb7b-69e2-4baf-af03-9be2f556dc6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:29.279802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dfb7b-69e2-4baf-af03-9be2f556dc6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:44:29.281266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dfb7b-69e2-4baf-af03-9be2f556dc6f service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:44:29.291224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 751/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:44:31.552687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103ecb-0ea8-4054-8fc6-e03f6576b154 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:31.553243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103ecb-0ea8-4054-8fc6-e03f6576b154 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:31.553375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103ecb-0ea8-4054-8fc6-e03f6576b154 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:31.555332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 737/1488] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:44:31.560616 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a518b4-9c25-4ac9-a410-d22bfaad2f11 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:31.561022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a518b4-9c25-4ac9-a410-d22bfaad2f11 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:31.561176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a518b4-9c25-4ac9-a410-d22bfaad2f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:31.578662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-d8a518b4-9c25-4ac9-a410-d22bfaad2f11 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:44:31.579877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d8a518b4-9c25-4ac9-a410-d22bfaad2f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:44:31.587641 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 752/1489] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:44:31.627775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103ecb-0ea8-4054-8fc6-e03f6576b154 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:44:31.638581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103ecb-0ea8-4054-8fc6-e03f6576b154 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:44:31.642342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d252a5-3854-4417-88fd-ca6a412fe192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:44:31.643032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d252a5-3854-4417-88fd-ca6a412fe192 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:31.643320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d252a5-3854-4417-88fd-ca6a412fe192 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:31.643599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d252a5-3854-4417-88fd-ca6a412fe192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:31.652384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d252a5-3854-4417-88fd-ca6a412fe192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:44:31.653800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d252a5-3854-4417-88fd-ca6a412fe192 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:44:31.663566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 738/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:44:33.925159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c9ff1-b52a-4fda-9643-7ccb2dd43b93 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:33.925709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c9ff1-b52a-4fda-9643-7ccb2dd43b93 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:33.925709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c9ff1-b52a-4fda-9643-7ccb2dd43b93 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:33.927111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 753/1491] 10.222.0.62 () {54 vars in 926 bytes} [Wed Jun 15 15:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:44:33.934027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ed48e-f016-49f6-836b-c8d9746ba717 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:33.934460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ed48e-f016-49f6-836b-c8d9746ba717 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:33.934676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ed48e-f016-49f6-836b-c8d9746ba717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:33.959266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-cc5ed48e-f016-49f6-836b-c8d9746ba717 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:44:33.961344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cc5ed48e-f016-49f6-836b-c8d9746ba717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:44:33.972838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 739/1492] 10.222.0.62 () {58 vars in 1023 bytes} [Wed Jun 15 15:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:44:34.011176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c9ff1-b52a-4fda-9643-7ccb2dd43b93 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:44:34.022354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c9ff1-b52a-4fda-9643-7ccb2dd43b93 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:44:34.026448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1e902-0c44-4775-b36d-7702be3e9658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:44:34.027113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1e902-0c44-4775-b36d-7702be3e9658 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:44:34.027398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1e902-0c44-4775-b36d-7702be3e9658 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:44:34.027675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1e902-0c44-4775-b36d-7702be3e9658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:44:34.036477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1e902-0c44-4775-b36d-7702be3e9658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:44:34.037886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1e902-0c44-4775-b36d-7702be3e9658 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:44:34.047201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 754/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:44:43.954156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=33 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:44:44.043102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=33 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:46:04.624800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e912caa-6aef-40b6-a6d0-3b530d3ddbb3 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:04.625406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e912caa-6aef-40b6-a6d0-3b530d3ddbb3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:04.625406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e912caa-6aef-40b6-a6d0-3b530d3ddbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:04.643241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8e912caa-6aef-40b6-a6d0-3b530d3ddbb3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:46:04.644434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8e912caa-6aef-40b6-a6d0-3b530d3ddbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:46:04.652440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 740/1494] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:46:04.661492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ead65-6efc-48e7-b72e-29860e23012a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:04.661967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ead65-6efc-48e7-b72e-29860e23012a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:04.662077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ead65-6efc-48e7-b72e-29860e23012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:04.679996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-cf9ead65-6efc-48e7-b72e-29860e23012a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:46:04.681179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cf9ead65-6efc-48e7-b72e-29860e23012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:46:04.689578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 755/1495] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:46:04.693849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e912caa-6aef-40b6-a6d0-3b530d3ddbb3 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:04.703334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e912caa-6aef-40b6-a6d0-3b530d3ddbb3 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:04.707745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:04.708389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:04.708674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:04.708954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:04.709671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:46:04.709978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:04.711018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING py.warnings [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:46:04.711018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: warnings.warn(msg) Jun 15 15:46:04.711018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: Jun 15 15:46:04.711887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:04.723194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 741/1496] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:46:04.730979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ead65-6efc-48e7-b72e-29860e23012a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:04.740549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ead65-6efc-48e7-b72e-29860e23012a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:04.745251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dffc9a-8e38-40eb-9ab8-cb1ad8c4b851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:04.745889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dffc9a-8e38-40eb-9ab8-cb1ad8c4b851 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:04.746176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dffc9a-8e38-40eb-9ab8-cb1ad8c4b851 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:04.746452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dffc9a-8e38-40eb-9ab8-cb1ad8c4b851 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:04.747159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dffc9a-8e38-40eb-9ab8-cb1ad8c4b851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:04.749549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: WARNING py.warnings [None req-37dffc9a-8e38-40eb-9ab8-cb1ad8c4b851 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:46:04.749549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: warnings.warn(msg) Jun 15 15:46:04.749549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: Jun 15 15:46:04.750717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dffc9a-8e38-40eb-9ab8-cb1ad8c4b851 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:04.785415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 756/1497] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:46:04 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:46:04.789498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:04.799857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ac8ee-4837-40f3-8343-7bde5a20fac1 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:04.854454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78eecb-b7e2-4729-8775-2c66f703fe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:04.855144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78eecb-b7e2-4729-8775-2c66f703fe0a admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:04.855431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78eecb-b7e2-4729-8775-2c66f703fe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:04.855716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78eecb-b7e2-4729-8775-2c66f703fe0a admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:04.856404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78eecb-b7e2-4729-8775-2c66f703fe0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:04.857962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING py.warnings [None req-3f78eecb-b7e2-4729-8775-2c66f703fe0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:46:04.857962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: warnings.warn(msg) Jun 15 15:46:04.857962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: Jun 15 15:46:04.859188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78eecb-b7e2-4729-8775-2c66f703fe0a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:04.878296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 742/1498] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:46:04 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:46:04.882349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dffc9a-8e38-40eb-9ab8-cb1ad8c4b851 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:04.892338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dffc9a-8e38-40eb-9ab8-cb1ad8c4b851 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:04.896790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef838b-e2e0-4bd1-b840-f29cfd7b6385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:04.897442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef838b-e2e0-4bd1-b840-f29cfd7b6385 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:04.897727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef838b-e2e0-4bd1-b840-f29cfd7b6385 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:04.898002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef838b-e2e0-4bd1-b840-f29cfd7b6385 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:04.898719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef838b-e2e0-4bd1-b840-f29cfd7b6385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:04.899753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: WARNING py.warnings [None req-38ef838b-e2e0-4bd1-b840-f29cfd7b6385 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:46:04.899753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: warnings.warn(msg) Jun 15 15:46:04.899753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: Jun 15 15:46:04.900623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef838b-e2e0-4bd1-b840-f29cfd7b6385 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:04.914064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 757/1499] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:46:04.917978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78eecb-b7e2-4729-8775-2c66f703fe0a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:04.928358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78eecb-b7e2-4729-8775-2c66f703fe0a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:04.932873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99ea23-fab3-49b9-a0e9-1cc6610ad537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:04.933526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99ea23-fab3-49b9-a0e9-1cc6610ad537 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:04.933812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99ea23-fab3-49b9-a0e9-1cc6610ad537 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:04.934094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99ea23-fab3-49b9-a0e9-1cc6610ad537 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:04.934865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99ea23-fab3-49b9-a0e9-1cc6610ad537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:04.935853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING py.warnings [None req-5b99ea23-fab3-49b9-a0e9-1cc6610ad537 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:46:04.935853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: warnings.warn(msg) Jun 15 15:46:04.935853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: Jun 15 15:46:04.936724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99ea23-fab3-49b9-a0e9-1cc6610ad537 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:04.949817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 743/1500] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:46:04.953882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef838b-e2e0-4bd1-b840-f29cfd7b6385 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:04.963804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef838b-e2e0-4bd1-b840-f29cfd7b6385 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:04.968814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff801838-94f3-4163-ab9f-db18a9e328f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:04.969689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801838-94f3-4163-ab9f-db18a9e328f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:04.969972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801838-94f3-4163-ab9f-db18a9e328f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:04.970251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801838-94f3-4163-ab9f-db18a9e328f1 admin admin] PATH_INFO: `/v3/projects/028c7c1e9d6a4572833be74a0632b0af/users/9775a0000e03438f83449d89cdd1521a/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:04.986090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff801838-94f3-4163-ab9f-db18a9e328f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=028c7c1e9d6a4572833be74a0632b0af, user_id=9775a0000e03438f83449d89cdd1521a, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:04.987153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: WARNING py.warnings [None req-ff801838-94f3-4163-ab9f-db18a9e328f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:46:04.987153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: warnings.warn(msg) Jun 15 15:46:04.987153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: Jun 15 15:46:04.989062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff801838-94f3-4163-ab9f-db18a9e328f1 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:05.001816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 758/1501] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:46:04 2022] PUT /identity/v3/projects/028c7c1e9d6a4572833be74a0632b0af/users/9775a0000e03438f83449d89cdd1521a/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:46:05.012822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6d89e-d3c4-4172-b2d1-ec2d8210ff1c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:05.013161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6d89e-d3c4-4172-b2d1-ec2d8210ff1c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:05.013448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6d89e-d3c4-4172-b2d1-ec2d8210ff1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:05.039958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-81f6d89e-d3c4-4172-b2d1-ec2d8210ff1c None None] MFA Rules not processed for user `9775a0000e03438f83449d89cdd1521a`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:46:05.041169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-81f6d89e-d3c4-4172-b2d1-ec2d8210ff1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:46:05.103024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 744/1502] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:46:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:46:05.116666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761ffce-391a-477b-b196-f8c1f446d180 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:05.116963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761ffce-391a-477b-b196-f8c1f446d180 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:05.117250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761ffce-391a-477b-b196-f8c1f446d180 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:05.118607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 759/1503] 10.222.0.62 () {54 vars in 963 bytes} [Wed Jun 15 15:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:46:05.123315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4496f1-f546-4b7c-b265-889dd0fa5776 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:05.123614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4496f1-f546-4b7c-b265-889dd0fa5776 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:05.123905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4496f1-f546-4b7c-b265-889dd0fa5776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:05.142354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0f4496f1-f546-4b7c-b265-889dd0fa5776 None None] MFA Rules not processed for user `1bd66c1b7c7147a281ef2e7253224069`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:46:05.143579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0f4496f1-f546-4b7c-b265-889dd0fa5776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:46:05.186776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:46:05.195559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 745/1504] 10.222.0.62 () {58 vars in 1060 bytes} [Wed Jun 15 15:46:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:46:05.203660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761ffce-391a-477b-b196-f8c1f446d180 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:05.214110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761ffce-391a-477b-b196-f8c1f446d180 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:05.218189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e251c-015f-46a2-8341-505754414932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:05.218858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e251c-015f-46a2-8341-505754414932 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:05.219142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e251c-015f-46a2-8341-505754414932 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:05.219418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e251c-015f-46a2-8341-505754414932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:05.227550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e251c-015f-46a2-8341-505754414932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:05.228967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e251c-015f-46a2-8341-505754414932 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:05.237600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 760/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:46:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:46:05.279904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae8254-b398-4aea-87d4-8a92e48d6daa None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:05.280193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae8254-b398-4aea-87d4-8a92e48d6daa None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:05.280473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae8254-b398-4aea-87d4-8a92e48d6daa None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:05.281739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 746/1506] 10.222.0.62 () {52 vars in 829 bytes} [Wed Jun 15 15:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:46:05.285362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e251c-015f-46a2-8341-505754414932 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:05.350211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e251c-015f-46a2-8341-505754414932 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:05.354687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:05.355585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:05.355871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:05.356149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] PATH_INFO: `/v3/users/9775a0000e03438f83449d89cdd1521a` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:05.357349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9775a0000e03438f83449d89cdd1521a)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:05.358408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: WARNING py.warnings [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:46:05.358408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: warnings.warn(msg) Jun 15 15:46:05.358408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: Jun 15 15:46:05.359640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:05.422347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9775a0000e03438f83449d89cdd1521a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:05.431073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9775a0000e03438f83449d89cdd1521a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:05.443245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9775a0000e03438f83449d89cdd1521a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:05.444013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9775a0000e03438f83449d89cdd1521a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:05.460589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:46:05.463454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9775a0000e03438f83449d89cdd1521a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:05.471748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 761/1507] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:46:05 2022] DELETE /identity/v3/users/9775a0000e03438f83449d89cdd1521a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:46:05.476092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce9d8-9113-46b5-bfd4-a28bad7c382f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:05.476383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce9d8-9113-46b5-bfd4-a28bad7c382f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:05.476667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce9d8-9113-46b5-bfd4-a28bad7c382f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:05.493425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9b0ce9d8-9113-46b5-bfd4-a28bad7c382f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:46:05.494592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9b0ce9d8-9113-46b5-bfd4-a28bad7c382f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:46:05.552276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 747/1508] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:46:05.562192 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:05.563620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:46:05.618387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79401c2d-f206-4169-88b5-da2b74698cb1 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:05.621956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a8bd0-73cb-43c8-aeb7-43f2b2d7bce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:05.623156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a8bd0-73cb-43c8-aeb7-43f2b2d7bce8 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:05.623436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a8bd0-73cb-43c8-aeb7-43f2b2d7bce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:05.623714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a8bd0-73cb-43c8-aeb7-43f2b2d7bce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:05.632621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a8bd0-73cb-43c8-aeb7-43f2b2d7bce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:05.634104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a8bd0-73cb-43c8-aeb7-43f2b2d7bce8 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:05.644764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 762/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:46:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:46:06.010373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ce9d8-9113-46b5-bfd4-a28bad7c382f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:46:06.011867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9b0ce9d8-9113-46b5-bfd4-a28bad7c382f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:46:06.023276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ce9d8-9113-46b5-bfd4-a28bad7c382f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:46:06.028917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:46:06.030377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:46:06.030876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:46:06.031164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] PATH_INFO: `/v3/projects/028c7c1e9d6a4572833be74a0632b0af` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:46:06.032708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=028c7c1e9d6a4572833be74a0632b0af)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:46:06.033459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING py.warnings [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:46:06.033459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: warnings.warn(msg) Jun 15 15:46:06.033459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: Jun 15 15:46:06.035551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:46:06.095936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '028c7c1e9d6a4572833be74a0632b0af', 'request_id': 'req-8f117d31-0c29-4940-b822-90e209e78395'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:06.109619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '028c7c1e9d6a4572833be74a0632b0af', 'request_id': 'req-8f117d31-0c29-4940-b822-90e209e78395'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:06.119284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '028c7c1e9d6a4572833be74a0632b0af', 'request_id': 'req-8f117d31-0c29-4940-b822-90e209e78395'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:06.130703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] The token cache is being invalidate because project 028c7c1e9d6a4572833be74a0632b0af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:46:06.131027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:06.131663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f117d31-0c29-4940-b822-90e209e78395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:46:06.133282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 748/1510] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:46:06 2022] DELETE /identity/v3/projects/028c7c1e9d6a4572833be74a0632b0af => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:46:15.640398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:46:16.127679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:47:50.379233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301d4f9-e211-4671-aa1c-99a871dd331f None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:47:50.379836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301d4f9-e211-4671-aa1c-99a871dd331f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:47:50.379836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301d4f9-e211-4671-aa1c-99a871dd331f None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:47:50.381162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:47:50.386072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a8de-b854-48a9-944b-579ffbd07008 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:47:50.386526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a8de-b854-48a9-944b-579ffbd07008 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:47:50.386634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a8de-b854-48a9-944b-579ffbd07008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:47:50.404853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f5a4a8de-b854-48a9-944b-579ffbd07008 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:47:50.406060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f5a4a8de-b854-48a9-944b-579ffbd07008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:47:50.464796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:47:50.639649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301d4f9-e211-4671-aa1c-99a871dd331f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:47:50.641350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1301d4f9-e211-4671-aa1c-99a871dd331f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:47:50.715505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301d4f9-e211-4671-aa1c-99a871dd331f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:47:50.721695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242e618-cce4-4ede-b92c-389057e4f538 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:47:50.722350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e618-cce4-4ede-b92c-389057e4f538 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:47:50.722637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e618-cce4-4ede-b92c-389057e4f538 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:47:50.722941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e618-cce4-4ede-b92c-389057e4f538 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:47:50.731356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242e618-cce4-4ede-b92c-389057e4f538 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:47:50.732970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242e618-cce4-4ede-b92c-389057e4f538 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:47:50.742325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 764/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:47:51.611329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fc896-c7cf-41af-93e2-b154c817d2cb None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:47:51.611884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fc896-c7cf-41af-93e2-b154c817d2cb None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:47:51.611884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fc896-c7cf-41af-93e2-b154c817d2cb None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:47:51.613305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:47:51.618234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2bfd4-23e8-4595-9196-8af0bc418885 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:47:51.618662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2bfd4-23e8-4595-9196-8af0bc418885 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:47:51.618800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2bfd4-23e8-4595-9196-8af0bc418885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:47:51.636851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-86b2bfd4-23e8-4595-9196-8af0bc418885 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:47:51.638032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-86b2bfd4-23e8-4595-9196-8af0bc418885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:47:51.645652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:47:51.797407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fc896-c7cf-41af-93e2-b154c817d2cb None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:47:51.812177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fc896-c7cf-41af-93e2-b154c817d2cb None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:47:51.817528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacdc04-045d-47cf-bfaa-7a60b304ce7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:47:51.818142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacdc04-045d-47cf-bfaa-7a60b304ce7a service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:47:51.818425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacdc04-045d-47cf-bfaa-7a60b304ce7a service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:47:51.818703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacdc04-045d-47cf-bfaa-7a60b304ce7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:47:51.826701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacdc04-045d-47cf-bfaa-7a60b304ce7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:47:51.828327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacdc04-045d-47cf-bfaa-7a60b304ce7a service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:47:51.837484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 751/1516] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:47:54.786284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e029-aded-4a2e-9f11-bf574e471aaa None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:47:54.786834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e029-aded-4a2e-9f11-bf574e471aaa None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:47:54.786834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e029-aded-4a2e-9f11-bf574e471aaa None None] PATH_INFO: `/v3` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:47:54.788087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 766/1517] 10.222.0.81 () {54 vars in 925 bytes} [Wed Jun 15 15:47:54 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:47:54.796756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b9c28-c890-43f9-b9f2-333e63fd19cd None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:47:54.797198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b9c28-c890-43f9-b9f2-333e63fd19cd None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:47:54.797333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b9c28-c890-43f9-b9f2-333e63fd19cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:47:54.815500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-478b9c28-c890-43f9-b9f2-333e63fd19cd None None] MFA Rules not processed for user `a786a95c48c348dbba269e0e33265271`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:47:54.816709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-478b9c28-c890-43f9-b9f2-333e63fd19cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:47:54.877716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 752/1518] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Jun 15 15:47:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:48:01.631718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=37 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:48:03.128240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd854c49-d27d-4425-8e3a-03f546b8f513 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:03.128890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd854c49-d27d-4425-8e3a-03f546b8f513 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:03.128890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd854c49-d27d-4425-8e3a-03f546b8f513 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:03.130135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 767/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:48:03.134928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304cee-f5f7-4862-be3a-3fd328060951 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:03.135360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304cee-f5f7-4862-be3a-3fd328060951 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:03.135499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304cee-f5f7-4862-be3a-3fd328060951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:03.153235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8d304cee-f5f7-4862-be3a-3fd328060951 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:48:03.154435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8d304cee-f5f7-4862-be3a-3fd328060951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:03.162577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 753/1520] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:48:03.316411 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd854c49-d27d-4425-8e3a-03f546b8f513 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:48:03.330856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd854c49-d27d-4425-8e3a-03f546b8f513 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:48:03.336086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22525eff-4ff3-4107-9ff1-0e5c2509a070 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:48:03.336713 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22525eff-4ff3-4107-9ff1-0e5c2509a070 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:03.336992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22525eff-4ff3-4107-9ff1-0e5c2509a070 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:03.337266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22525eff-4ff3-4107-9ff1-0e5c2509a070 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:03.345144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22525eff-4ff3-4107-9ff1-0e5c2509a070 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:48:03.346715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22525eff-4ff3-4107-9ff1-0e5c2509a070 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:48:03.355864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 768/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:48:04.227659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dc06-75a1-4ed8-a188-820ca4342b7d None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:04.228233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dc06-75a1-4ed8-a188-820ca4342b7d None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:04.228233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dc06-75a1-4ed8-a188-820ca4342b7d None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:04.229599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 15:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:48:04.235473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b57cef-7f07-4306-8047-9de8c9df1b35 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:04.235925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b57cef-7f07-4306-8047-9de8c9df1b35 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:04.236115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b57cef-7f07-4306-8047-9de8c9df1b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:04.254625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-51b57cef-7f07-4306-8047-9de8c9df1b35 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:48:04.255872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-51b57cef-7f07-4306-8047-9de8c9df1b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:04.263721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:48:04.417743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0dc06-75a1-4ed8-a188-820ca4342b7d None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:48:04.432814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0dc06-75a1-4ed8-a188-820ca4342b7d None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:48:04.438191 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be92a1a-6b12-4882-b646-e63aa3baf1ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:48:04.438868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be92a1a-6b12-4882-b646-e63aa3baf1ed service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:04.439194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be92a1a-6b12-4882-b646-e63aa3baf1ed service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:04.439474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be92a1a-6b12-4882-b646-e63aa3baf1ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:04.449297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be92a1a-6b12-4882-b646-e63aa3baf1ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:48:04.451006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be92a1a-6b12-4882-b646-e63aa3baf1ed service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:48:04.460662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 755/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:48:04.562135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f414a1-866a-4ba2-9830-febcaded415e None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:04.562421 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f414a1-866a-4ba2-9830-febcaded415e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:04.562701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f414a1-866a-4ba2-9830-febcaded415e None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:04.564325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 770/1525] 10.222.0.62 () {54 vars in 960 bytes} [Wed Jun 15 15:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:48:04.574295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e7152-cc81-4c62-8fe7-d8d4990b53ea None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:04.574587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e7152-cc81-4c62-8fe7-d8d4990b53ea None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:04.574904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e7152-cc81-4c62-8fe7-d8d4990b53ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:04.594346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f76e7152-cc81-4c62-8fe7-d8d4990b53ea None None] MFA Rules not processed for user `a786a95c48c348dbba269e0e33265271`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:48:04.595593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f76e7152-cc81-4c62-8fe7-d8d4990b53ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:04.605261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 756/1526] 10.222.0.62 () {58 vars in 1057 bytes} [Wed Jun 15 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:48:04.613373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f414a1-866a-4ba2-9830-febcaded415e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:48:04.627158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f414a1-866a-4ba2-9830-febcaded415e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:48:04.632135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d363c-9a61-4284-af37-efe7a44b3475 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:48:04.632767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d363c-9a61-4284-af37-efe7a44b3475 service placement] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:04.633049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d363c-9a61-4284-af37-efe7a44b3475 service placement] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:04.633320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d363c-9a61-4284-af37-efe7a44b3475 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:04.642023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d363c-9a61-4284-af37-efe7a44b3475 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:48:04.643623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d363c-9a61-4284-af37-efe7a44b3475 service placement] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:48:04.653062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 771/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:48:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:48:08.503629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dcfab-59e5-406f-af1b-2b730f04f520 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:08.504161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dcfab-59e5-406f-af1b-2b730f04f520 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:08.504161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dcfab-59e5-406f-af1b-2b730f04f520 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:08.505538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:48:08.510714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05191dfc-e969-4306-b851-f50df724c50b None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:08.511131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05191dfc-e969-4306-b851-f50df724c50b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:08.511307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05191dfc-e969-4306-b851-f50df724c50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:08.528942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-05191dfc-e969-4306-b851-f50df724c50b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:48:08.530086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-05191dfc-e969-4306-b851-f50df724c50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:08.538107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:48:08.574550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4dcfab-59e5-406f-af1b-2b730f04f520 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:48:08.576237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cf4dcfab-59e5-406f-af1b-2b730f04f520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:08.632386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4dcfab-59e5-406f-af1b-2b730f04f520 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:48:08.635833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991d31-ad34-487c-ab5a-0f68886b12a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:48:08.636463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991d31-ad34-487c-ab5a-0f68886b12a3 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:08.636744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991d31-ad34-487c-ab5a-0f68886b12a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:08.637020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991d31-ad34-487c-ab5a-0f68886b12a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:08.645702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991d31-ad34-487c-ab5a-0f68886b12a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:48:08.647149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991d31-ad34-487c-ab5a-0f68886b12a3 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:48:08.651924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:48:08.657212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 758/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:48:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:48:10.756299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ad865-e980-4744-bdcd-7c7217de3994 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:10.756848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ad865-e980-4744-bdcd-7c7217de3994 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:10.756848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ad865-e980-4744-bdcd-7c7217de3994 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:10.758148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:48:10.763377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e963a-778c-4567-8780-fcfd5a6e0e36 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:10.763809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e963a-778c-4567-8780-fcfd5a6e0e36 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:10.763914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e963a-778c-4567-8780-fcfd5a6e0e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:10.784519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-673e963a-778c-4567-8780-fcfd5a6e0e36 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:48:10.784519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-673e963a-778c-4567-8780-fcfd5a6e0e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:10.792864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:48:10.831974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ad865-e980-4744-bdcd-7c7217de3994 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:48:10.833787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8e6ad865-e980-4744-bdcd-7c7217de3994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:10.847166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ad865-e980-4744-bdcd-7c7217de3994 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:48:10.850603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdecf4-dcb0-4e3a-9236-db24b8d6a8d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:48:10.851263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdecf4-dcb0-4e3a-9236-db24b8d6a8d2 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:10.851544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdecf4-dcb0-4e3a-9236-db24b8d6a8d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:10.851816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdecf4-dcb0-4e3a-9236-db24b8d6a8d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:10.860488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdecf4-dcb0-4e3a-9236-db24b8d6a8d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:48:10.861892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdecf4-dcb0-4e3a-9236-db24b8d6a8d2 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:48:10.871053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 774/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:48:14.451172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4220-6b2f-4666-a9b6-bc74796e678b None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:14.451592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4220-6b2f-4666-a9b6-bc74796e678b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:14.451728 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4220-6b2f-4666-a9b6-bc74796e678b None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:14.453625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:48:14.458721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4276-5b89-470b-81b1-902acd1ca8d3 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:14.459116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4276-5b89-470b-81b1-902acd1ca8d3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:14.459323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4276-5b89-470b-81b1-902acd1ca8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:14.479645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4bbd4276-5b89-470b-81b1-902acd1ca8d3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:48:14.480852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4bbd4276-5b89-470b-81b1-902acd1ca8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:14.489122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:48:14.758949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe4220-6b2f-4666-a9b6-bc74796e678b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:48:14.760647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-71fe4220-6b2f-4666-a9b6-bc74796e678b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:14.817104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe4220-6b2f-4666-a9b6-bc74796e678b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:48:14.820599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970e9e1-5d02-4f15-8680-728a149258e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:48:14.821235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970e9e1-5d02-4f15-8680-728a149258e0 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:14.821518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970e9e1-5d02-4f15-8680-728a149258e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:14.821796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970e9e1-5d02-4f15-8680-728a149258e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:14.834322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970e9e1-5d02-4f15-8680-728a149258e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:48:14.836069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970e9e1-5d02-4f15-8680-728a149258e0 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:48:14.847308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 761/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:48:16.986935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a9ca8-33df-4b1f-a887-42e0c4431b4a None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:16.987473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a9ca8-33df-4b1f-a887-42e0c4431b4a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:16.987473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a9ca8-33df-4b1f-a887-42e0c4431b4a None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:16.988865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 15:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:48:16.994269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0854ba-a3e4-40fd-ad29-5c6709203be0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:16.994649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0854ba-a3e4-40fd-ad29-5c6709203be0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:16.994870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0854ba-a3e4-40fd-ad29-5c6709203be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:17.012537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-6c0854ba-a3e4-40fd-ad29-5c6709203be0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:48:17.013729 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6c0854ba-a3e4-40fd-ad29-5c6709203be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:17.021506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:48:17.292234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a9ca8-33df-4b1f-a887-42e0c4431b4a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:48:17.293931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b79a9ca8-33df-4b1f-a887-42e0c4431b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:48:17.306926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a9ca8-33df-4b1f-a887-42e0c4431b4a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:48:17.310384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a77926-b2d0-4c32-9e96-50542da8d541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:48:17.311072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a77926-b2d0-4c32-9e96-50542da8d541 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:48:17.311348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a77926-b2d0-4c32-9e96-50542da8d541 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:48:17.311624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a77926-b2d0-4c32-9e96-50542da8d541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:48:17.320384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a77926-b2d0-4c32-9e96-50542da8d541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:48:17.321800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a77926-b2d0-4c32-9e96-50542da8d541 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:48:17.331575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 777/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:48:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:48:27.007529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=32 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:48:27.327049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=33 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:47.511291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef76aa-5b6a-4cbc-9797-b42af9e7ae80 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:47.511933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef76aa-5b6a-4cbc-9797-b42af9e7ae80 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:47.511933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef76aa-5b6a-4cbc-9797-b42af9e7ae80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:47.539785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-eaef76aa-5b6a-4cbc-9797-b42af9e7ae80 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:47.541001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-eaef76aa-5b6a-4cbc-9797-b42af9e7ae80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:47.551223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 763/1540] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:47.561861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac009e-8af2-4b44-8952-e0b8f9a8934c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:47.562558 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac009e-8af2-4b44-8952-e0b8f9a8934c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:47.562558 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac009e-8af2-4b44-8952-e0b8f9a8934c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:47.599571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-11ac009e-8af2-4b44-8952-e0b8f9a8934c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:47.601112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-11ac009e-8af2-4b44-8952-e0b8f9a8934c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:47.618062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 778/1541] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:47.623992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef76aa-5b6a-4cbc-9797-b42af9e7ae80 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:47.646824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef76aa-5b6a-4cbc-9797-b42af9e7ae80 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:47.654881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d138c57-8dd8-4bd6-962a-3a4f823643fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:47.655564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d138c57-8dd8-4bd6-962a-3a4f823643fb admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:47.655845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d138c57-8dd8-4bd6-962a-3a4f823643fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:47.656124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d138c57-8dd8-4bd6-962a-3a4f823643fb admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:47.658406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d138c57-8dd8-4bd6-962a-3a4f823643fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:49:47.658924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d138c57-8dd8-4bd6-962a-3a4f823643fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:47.661524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d138c57-8dd8-4bd6-962a-3a4f823643fb admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:47.686350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 764/1542] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:49:47.695246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac009e-8af2-4b44-8952-e0b8f9a8934c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:47.710081 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac009e-8af2-4b44-8952-e0b8f9a8934c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:47.722598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02234811-aacd-4fe7-8c01-ba97667202fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:47.723315 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234811-aacd-4fe7-8c01-ba97667202fd admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:47.723597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234811-aacd-4fe7-8c01-ba97667202fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:47.723876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02234811-aacd-4fe7-8c01-ba97667202fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:47.724602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02234811-aacd-4fe7-8c01-ba97667202fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:47.726578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02234811-aacd-4fe7-8c01-ba97667202fd admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:47.778373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 779/1543] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:47 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:47.783785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d138c57-8dd8-4bd6-962a-3a4f823643fb admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:47.799304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d138c57-8dd8-4bd6-962a-3a4f823643fb admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:47.888397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d2ff0-66cd-4bdf-b0e5-8cb0f9516b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:47.889090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d2ff0-66cd-4bdf-b0e5-8cb0f9516b4d admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:47.889372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d2ff0-66cd-4bdf-b0e5-8cb0f9516b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:47.889646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d2ff0-66cd-4bdf-b0e5-8cb0f9516b4d admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:47.890352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d2ff0-66cd-4bdf-b0e5-8cb0f9516b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:47.892876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d2ff0-66cd-4bdf-b0e5-8cb0f9516b4d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:47.926692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 765/1544] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:47 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:47.934044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02234811-aacd-4fe7-8c01-ba97667202fd admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:47.956902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02234811-aacd-4fe7-8c01-ba97667202fd admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:47.971545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8c002-c0a9-4ede-ad2a-ef9e840f8db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:47.972162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8c002-c0a9-4ede-ad2a-ef9e840f8db7 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:47.972448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8c002-c0a9-4ede-ad2a-ef9e840f8db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:47.972729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8c002-c0a9-4ede-ad2a-ef9e840f8db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:47.973461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8c002-c0a9-4ede-ad2a-ef9e840f8db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:47.975112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8c002-c0a9-4ede-ad2a-ef9e840f8db7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:47.993807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 780/1545] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:47.997829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d2ff0-66cd-4bdf-b0e5-8cb0f9516b4d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.012431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d2ff0-66cd-4bdf-b0e5-8cb0f9516b4d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.022839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ca1ff-2705-4460-a91b-3f6ab976eeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.023619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca1ff-2705-4460-a91b-3f6ab976eeae admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.023970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca1ff-2705-4460-a91b-3f6ab976eeae admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.024246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca1ff-2705-4460-a91b-3f6ab976eeae admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.025040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ca1ff-2705-4460-a91b-3f6ab976eeae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.026644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ca1ff-2705-4460-a91b-3f6ab976eeae admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.045857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 766/1546] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:48.050561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8c002-c0a9-4ede-ad2a-ef9e840f8db7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.065161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8c002-c0a9-4ede-ad2a-ef9e840f8db7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.080084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07b7f1-cd33-4b6d-bf32-6b15e983bdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.081685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07b7f1-cd33-4b6d-bf32-6b15e983bdec admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.082156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07b7f1-cd33-4b6d-bf32-6b15e983bdec admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.082612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07b7f1-cd33-4b6d-bf32-6b15e983bdec admin admin] PATH_INFO: `/v3/projects/669dc76c39b3427b8c55fd47de62595f/users/434c81b82dee404f8d143f1bdf56d3ca/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.105564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07b7f1-cd33-4b6d-bf32-6b15e983bdec admin admin] RBAC: Authorizing `identity:create_grant(project_id=669dc76c39b3427b8c55fd47de62595f, user_id=434c81b82dee404f8d143f1bdf56d3ca, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.109026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07b7f1-cd33-4b6d-bf32-6b15e983bdec admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.112895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43c10d-c7b4-4237-8eae-77471efe5c16 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.113342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43c10d-c7b4-4237-8eae-77471efe5c16 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.113636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43c10d-c7b4-4237-8eae-77471efe5c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.145840 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 781/1547] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:48 2022] PUT /identity/v3/projects/669dc76c39b3427b8c55fd47de62595f/users/434c81b82dee404f8d143f1bdf56d3ca/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:48.147473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ecdce-c561-4e8d-80b8-0d14f4fa840d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.147766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ecdce-c561-4e8d-80b8-0d14f4fa840d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.148047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ecdce-c561-4e8d-80b8-0d14f4fa840d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.150991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ea43c10d-c7b4-4237-8eae-77471efe5c16 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.152231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ea43c10d-c7b4-4237-8eae-77471efe5c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.186637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-3a9ecdce-c561-4e8d-80b8-0d14f4fa840d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.188479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3a9ecdce-c561-4e8d-80b8-0d14f4fa840d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.288761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 767/1548] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.290289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4a5be-1afb-4caf-a75b-7f9d0b92e49a None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.290581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4a5be-1afb-4caf-a75b-7f9d0b92e49a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.290905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4a5be-1afb-4caf-a75b-7f9d0b92e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.324384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-92c4a5be-1afb-4caf-a75b-7f9d0b92e49a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.325605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-92c4a5be-1afb-4caf-a75b-7f9d0b92e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.332150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 782/1549] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.333724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a50a3a-f1ee-418d-926c-1c633f0ca47f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.334195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a50a3a-f1ee-418d-926c-1c633f0ca47f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.334643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a50a3a-f1ee-418d-926c-1c633f0ca47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.348460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 768/1550] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.349781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e41d9-1a10-4a61-9241-956712a884cd None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.350069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e41d9-1a10-4a61-9241-956712a884cd None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.350349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e41d9-1a10-4a61-9241-956712a884cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.374389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-08a50a3a-f1ee-418d-926c-1c633f0ca47f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.382509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-08a50a3a-f1ee-418d-926c-1c633f0ca47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.398320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-1c7e41d9-1a10-4a61-9241-956712a884cd None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.400438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1c7e41d9-1a10-4a61-9241-956712a884cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.402939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 783/1551] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.407644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e708e-861f-4be3-9ff8-36048b419e5a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.408251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e708e-861f-4be3-9ff8-36048b419e5a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.409125 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e708e-861f-4be3-9ff8-36048b419e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.416937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 769/1552] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.418275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a4642-81ed-4322-9bea-0e07dc5b153b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.418570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a4642-81ed-4322-9bea-0e07dc5b153b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.427224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a4642-81ed-4322-9bea-0e07dc5b153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.454163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d91a4642-81ed-4322-9bea-0e07dc5b153b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.454163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d91a4642-81ed-4322-9bea-0e07dc5b153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.463064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-338e708e-861f-4be3-9ff8-36048b419e5a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.464802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-338e708e-861f-4be3-9ff8-36048b419e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.465890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 770/1553] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.466769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a4642-81ed-4322-9bea-0e07dc5b153b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.480676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 784/1554] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.485141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e708e-861f-4be3-9ff8-36048b419e5a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.495908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e708e-861f-4be3-9ff8-36048b419e5a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.502691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b20b26-1bf1-421b-99bc-2d93a8c90514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.503499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b20b26-1bf1-421b-99bc-2d93a8c90514 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.503888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b20b26-1bf1-421b-99bc-2d93a8c90514 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.504298 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b20b26-1bf1-421b-99bc-2d93a8c90514 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.505138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b20b26-1bf1-421b-99bc-2d93a8c90514 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:49:48.505547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b20b26-1bf1-421b-99bc-2d93a8c90514 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.506672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: WARNING py.warnings [None req-c1b20b26-1bf1-421b-99bc-2d93a8c90514 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:49:48.506672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: warnings.warn(msg) Jun 15 15:49:48.506672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: Jun 15 15:49:48.507862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b20b26-1bf1-421b-99bc-2d93a8c90514 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.517004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:48.521678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 785/1555] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:49:48.527086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f09ef-3916-49a7-ac53-40913a69884e None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.527384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f09ef-3916-49a7-ac53-40913a69884e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.527662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f09ef-3916-49a7-ac53-40913a69884e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.538261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a4642-81ed-4322-9bea-0e07dc5b153b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.541960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a2f01-e5d5-40dc-865f-1d6da4f42100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.542548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2f01-e5d5-40dc-865f-1d6da4f42100 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.542877 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2f01-e5d5-40dc-865f-1d6da4f42100 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.543163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2f01-e5d5-40dc-865f-1d6da4f42100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.547208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-108f09ef-3916-49a7-ac53-40913a69884e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.548408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-108f09ef-3916-49a7-ac53-40913a69884e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.553259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a2f01-e5d5-40dc-865f-1d6da4f42100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.554853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a2f01-e5d5-40dc-865f-1d6da4f42100 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.557429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 786/1556] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.558905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890736b-f7a3-4049-8908-8c15255cbef7 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.559199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890736b-f7a3-4049-8908-8c15255cbef7 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.561904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890736b-f7a3-4049-8908-8c15255cbef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.571923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 771/1557] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:48.573214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d8ba-127f-466a-8e66-36fa5e190f70 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.573511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d8ba-127f-466a-8e66-36fa5e190f70 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.573794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d8ba-127f-466a-8e66-36fa5e190f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.581368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-7890736b-f7a3-4049-8908-8c15255cbef7 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.583380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7890736b-f7a3-4049-8908-8c15255cbef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.592270 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 787/1558] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.593222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890736b-f7a3-4049-8908-8c15255cbef7 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.595004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-53d8d8ba-127f-466a-8e66-36fa5e190f70 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.596198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-53d8d8ba-127f-466a-8e66-36fa5e190f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.602886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890736b-f7a3-4049-8908-8c15255cbef7 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.608254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 772/1559] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.608254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1785797-e01d-4037-a855-c078e7aa1887 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.608254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1785797-e01d-4037-a855-c078e7aa1887 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.608254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1785797-e01d-4037-a855-c078e7aa1887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.612184 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a4c53-3f71-43a9-9ff9-2a3f63e22635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.612938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a4c53-3f71-43a9-9ff9-2a3f63e22635 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.613222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a4c53-3f71-43a9-9ff9-2a3f63e22635 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.613496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a4c53-3f71-43a9-9ff9-2a3f63e22635 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.614244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a4c53-3f71-43a9-9ff9-2a3f63e22635 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:49:48.614598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a4c53-3f71-43a9-9ff9-2a3f63e22635 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.616395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a4c53-3f71-43a9-9ff9-2a3f63e22635 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.627506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c1785797-e01d-4037-a855-c078e7aa1887 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.628718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c1785797-e01d-4037-a855-c078e7aa1887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.636458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 788/1560] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:49:48.636458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a4c53-3f71-43a9-9ff9-2a3f63e22635 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.644507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 773/1561] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.645333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1785797-e01d-4037-a855-c078e7aa1887 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.655475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a4c53-3f71-43a9-9ff9-2a3f63e22635 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.661500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1785797-e01d-4037-a855-c078e7aa1887 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.661677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6be77-936c-47d0-9a84-ede3d19cfcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.662873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6be77-936c-47d0-9a84-ede3d19cfcfc admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.663329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6be77-936c-47d0-9a84-ede3d19cfcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.663792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6be77-936c-47d0-9a84-ede3d19cfcfc admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.664539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6be77-936c-47d0-9a84-ede3d19cfcfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.666604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6be77-936c-47d0-9a84-ede3d19cfcfc admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.666754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98983d2f-2678-4cbf-96eb-d870badb3c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.667416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98983d2f-2678-4cbf-96eb-d870badb3c59 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.668055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98983d2f-2678-4cbf-96eb-d870badb3c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.668486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98983d2f-2678-4cbf-96eb-d870badb3c59 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.669044 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98983d2f-2678-4cbf-96eb-d870badb3c59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:49:48.669617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98983d2f-2678-4cbf-96eb-d870badb3c59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.671238 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98983d2f-2678-4cbf-96eb-d870badb3c59 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.691923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 774/1562] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:49:48.691923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f176ff-5623-4179-b457-76c51d28ef6e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.691923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f176ff-5623-4179-b457-76c51d28ef6e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.691923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f176ff-5623-4179-b457-76c51d28ef6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.709617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 789/1563] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/projects => generated 389 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:48.709617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5eb53-d71e-4a56-8cfa-f37a370808fb None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.709617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5eb53-d71e-4a56-8cfa-f37a370808fb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.709617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5eb53-d71e-4a56-8cfa-f37a370808fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.711153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d0f176ff-5623-4179-b457-76c51d28ef6e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.712562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d0f176ff-5623-4179-b457-76c51d28ef6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.728144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-c4b5eb53-d71e-4a56-8cfa-f37a370808fb None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.729361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c4b5eb53-d71e-4a56-8cfa-f37a370808fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.752514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:48.793536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 775/1564] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.794137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f176ff-5623-4179-b457-76c51d28ef6e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.805008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f176ff-5623-4179-b457-76c51d28ef6e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.809912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cea9f-7247-4da4-be75-b3d741c83c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.810605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cea9f-7247-4da4-be75-b3d741c83c50 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.810947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cea9f-7247-4da4-be75-b3d741c83c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.811192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cea9f-7247-4da4-be75-b3d741c83c50 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.811913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cea9f-7247-4da4-be75-b3d741c83c50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:49:48.812223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cea9f-7247-4da4-be75-b3d741c83c50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.813485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 790/1565] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.814363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5eb53-d71e-4a56-8cfa-f37a370808fb None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.818296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cea9f-7247-4da4-be75-b3d741c83c50 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.824326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5eb53-d71e-4a56-8cfa-f37a370808fb None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.829764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b869eec-ab73-495a-9877-1c160a61d0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.830396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b869eec-ab73-495a-9877-1c160a61d0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.830684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b869eec-ab73-495a-9877-1c160a61d0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.830996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b869eec-ab73-495a-9877-1c160a61d0e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.831717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b869eec-ab73-495a-9877-1c160a61d0e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.833506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 776/1566] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:49:48.833624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b869eec-ab73-495a-9877-1c160a61d0e3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.841927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f07b9-d40c-45ba-a724-2d80cdea8208 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.842228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f07b9-d40c-45ba-a724-2d80cdea8208 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.842512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f07b9-d40c-45ba-a724-2d80cdea8208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.870190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 791/1567] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:48.870190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b869eec-ab73-495a-9877-1c160a61d0e3 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.874181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d06f07b9-d40c-45ba-a724-2d80cdea8208 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:48.876150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d06f07b9-d40c-45ba-a724-2d80cdea8208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:48.946777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b869eec-ab73-495a-9877-1c160a61d0e3 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.947111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 777/1568] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:48.948148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f07b9-d40c-45ba-a724-2d80cdea8208 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:48.957783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946f58c-4836-495d-85ee-f35bbcc0a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.958397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f07b9-d40c-45ba-a724-2d80cdea8208 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:48.958693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946f58c-4836-495d-85ee-f35bbcc0a221 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.959204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946f58c-4836-495d-85ee-f35bbcc0a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.959596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946f58c-4836-495d-85ee-f35bbcc0a221 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.960602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946f58c-4836-495d-85ee-f35bbcc0a221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.962982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946f58c-4836-495d-85ee-f35bbcc0a221 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.963787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9f8d1-09b1-481c-9133-94e3858ba794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:48.964586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9f8d1-09b1-481c-9133-94e3858ba794 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.964971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9f8d1-09b1-481c-9133-94e3858ba794 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.965306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9f8d1-09b1-481c-9133-94e3858ba794 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.966075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9f8d1-09b1-481c-9133-94e3858ba794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:48.968103 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9f8d1-09b1-481c-9133-94e3858ba794 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:48.993307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 778/1569] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:48.995136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d1606-e3c4-4682-8c67-118071145921 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:48.995390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d1606-e3c4-4682-8c67-118071145921 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:48.995753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d1606-e3c4-4682-8c67-118071145921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:48.998302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 792/1570] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.002159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946f58c-4836-495d-85ee-f35bbcc0a221 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.017796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-1b5d1606-e3c4-4682-8c67-118071145921 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:49.019146 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1b5d1606-e3c4-4682-8c67-118071145921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:49.049536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:49.086859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946f58c-4836-495d-85ee-f35bbcc0a221 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.092197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4454673-bc20-448e-9d4e-9399261e2ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.092973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4454673-bc20-448e-9d4e-9399261e2ddf admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.093328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4454673-bc20-448e-9d4e-9399261e2ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.094145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4454673-bc20-448e-9d4e-9399261e2ddf admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.094824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4454673-bc20-448e-9d4e-9399261e2ddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:49:49.095141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4454673-bc20-448e-9d4e-9399261e2ddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.096930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4454673-bc20-448e-9d4e-9399261e2ddf admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.113069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 779/1571] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:49.120654 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 793/1572] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:49:49.120654 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4454673-bc20-448e-9d4e-9399261e2ddf admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.122039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d1606-e3c4-4682-8c67-118071145921 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.126674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4454673-bc20-448e-9d4e-9399261e2ddf admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.131841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ae376-e28f-41bc-a148-de773fa9a2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.132561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ae376-e28f-41bc-a148-de773fa9a2fe admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.132846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ae376-e28f-41bc-a148-de773fa9a2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.133168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ae376-e28f-41bc-a148-de773fa9a2fe admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.133871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ae376-e28f-41bc-a148-de773fa9a2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.134918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: WARNING py.warnings [None req-9b8ae376-e28f-41bc-a148-de773fa9a2fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:49:49.134918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: warnings.warn(msg) Jun 15 15:49:49.134918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: Jun 15 15:49:49.136530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ae376-e28f-41bc-a148-de773fa9a2fe admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.141631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d1606-e3c4-4682-8c67-118071145921 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.148054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e853a47-b7be-4605-9a64-ac922ae3ec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.148914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e853a47-b7be-4605-9a64-ac922ae3ec15 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.149243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e853a47-b7be-4605-9a64-ac922ae3ec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.149460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e853a47-b7be-4605-9a64-ac922ae3ec15 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.150235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e853a47-b7be-4605-9a64-ac922ae3ec15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.151438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING py.warnings [None req-6e853a47-b7be-4605-9a64-ac922ae3ec15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:49:49.151438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: warnings.warn(msg) Jun 15 15:49:49.151438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: Jun 15 15:49:49.152770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e853a47-b7be-4605-9a64-ac922ae3ec15 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.166610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 794/1573] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/users => generated 290 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.167495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ae376-e28f-41bc-a148-de773fa9a2fe admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.174006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:49.181761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ae376-e28f-41bc-a148-de773fa9a2fe admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.186817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 780/1574] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.188143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e853a47-b7be-4605-9a64-ac922ae3ec15 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.191011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3597a-b480-4b04-936d-bcd1bd066875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.192701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3597a-b480-4b04-936d-bcd1bd066875 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.193249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3597a-b480-4b04-936d-bcd1bd066875 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.193540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3597a-b480-4b04-936d-bcd1bd066875 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.194294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3597a-b480-4b04-936d-bcd1bd066875 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:49:49.194637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3597a-b480-4b04-936d-bcd1bd066875 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.196293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3597a-b480-4b04-936d-bcd1bd066875 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.212051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 795/1575] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:49:49.213636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3597a-b480-4b04-936d-bcd1bd066875 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.227329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3597a-b480-4b04-936d-bcd1bd066875 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.265881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e853a47-b7be-4605-9a64-ac922ae3ec15 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.272862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e234592-c6ca-448e-9861-19cb7f937ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.273550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e234592-c6ca-448e-9861-19cb7f937ccc admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.273836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e234592-c6ca-448e-9861-19cb7f937ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.274120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e234592-c6ca-448e-9861-19cb7f937ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.280200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e234592-c6ca-448e-9861-19cb7f937ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.282392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e234592-c6ca-448e-9861-19cb7f937ccc admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.300969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 781/1576] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:49.302112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e234592-c6ca-448e-9861-19cb7f937ccc admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.305356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5cd6a-8ef9-4248-8eac-7787034dde83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.306430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5cd6a-8ef9-4248-8eac-7787034dde83 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.306719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5cd6a-8ef9-4248-8eac-7787034dde83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.307180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5cd6a-8ef9-4248-8eac-7787034dde83 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.307750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5cd6a-8ef9-4248-8eac-7787034dde83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.310864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5cd6a-8ef9-4248-8eac-7787034dde83 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.315074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e234592-c6ca-448e-9861-19cb7f937ccc admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.322963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f1bb2-d7ca-42a8-9ecf-db587e96fbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.323897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f1bb2-d7ca-42a8-9ecf-db587e96fbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.324142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f1bb2-d7ca-42a8-9ecf-db587e96fbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.324463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f1bb2-d7ca-42a8-9ecf-db587e96fbe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.325803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f1bb2-d7ca-42a8-9ecf-db587e96fbe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:49:49.325803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f1bb2-d7ca-42a8-9ecf-db587e96fbe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.328406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f1bb2-d7ca-42a8-9ecf-db587e96fbe3 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.350178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 782/1577] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:49:49.350178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f1bb2-d7ca-42a8-9ecf-db587e96fbe3 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.353309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 796/1578] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.354146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5cd6a-8ef9-4248-8eac-7787034dde83 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.366160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f1bb2-d7ca-42a8-9ecf-db587e96fbe3 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.368022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5cd6a-8ef9-4248-8eac-7787034dde83 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.374601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c12b63-a60c-4c5e-aa2a-7c83de6ec004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.375466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e24d9a-b4ee-46e6-b690-8e7f9fffd8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.376075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c12b63-a60c-4c5e-aa2a-7c83de6ec004 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.376192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24d9a-b4ee-46e6-b690-8e7f9fffd8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.376354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c12b63-a60c-4c5e-aa2a-7c83de6ec004 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.376456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24d9a-b4ee-46e6-b690-8e7f9fffd8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.376732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24d9a-b4ee-46e6-b690-8e7f9fffd8e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.376890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c12b63-a60c-4c5e-aa2a-7c83de6ec004 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.377620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e24d9a-b4ee-46e6-b690-8e7f9fffd8e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.377905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c12b63-a60c-4c5e-aa2a-7c83de6ec004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.379587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e24d9a-b4ee-46e6-b690-8e7f9fffd8e7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.379774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c12b63-a60c-4c5e-aa2a-7c83de6ec004 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.394062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 797/1579] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:49.395128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c12b63-a60c-4c5e-aa2a-7c83de6ec004 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.408479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c12b63-a60c-4c5e-aa2a-7c83de6ec004 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.416502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d671922-f481-464b-84ee-21f33348e8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.417242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671922-f481-464b-84ee-21f33348e8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.417529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671922-f481-464b-84ee-21f33348e8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.417811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671922-f481-464b-84ee-21f33348e8b4 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.418509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d671922-f481-464b-84ee-21f33348e8b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.420801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d671922-f481-464b-84ee-21f33348e8b4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.439519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 783/1580] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.440223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e24d9a-b4ee-46e6-b690-8e7f9fffd8e7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.461166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 798/1581] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/users => generated 287 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.462236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d671922-f481-464b-84ee-21f33348e8b4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.480831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d671922-f481-464b-84ee-21f33348e8b4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.517453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e24d9a-b4ee-46e6-b690-8e7f9fffd8e7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.522502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8f59f-3f99-44f7-946f-60af9a374b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.523416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8f59f-3f99-44f7-946f-60af9a374b33 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.523773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8f59f-3f99-44f7-946f-60af9a374b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.524173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8f59f-3f99-44f7-946f-60af9a374b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.524947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8f59f-3f99-44f7-946f-60af9a374b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.527094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8f59f-3f99-44f7-946f-60af9a374b33 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.538849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:49.567615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 784/1582] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.568168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98810c8-d545-406e-98e2-0262ee508a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.568664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8f59f-3f99-44f7-946f-60af9a374b33 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.569786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98810c8-d545-406e-98e2-0262ee508a42 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.570138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98810c8-d545-406e-98e2-0262ee508a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.570690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98810c8-d545-406e-98e2-0262ee508a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.571696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98810c8-d545-406e-98e2-0262ee508a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.573446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98810c8-d545-406e-98e2-0262ee508a42 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.591054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 799/1583] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:49.591054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98810c8-d545-406e-98e2-0262ee508a42 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.602696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98810c8-d545-406e-98e2-0262ee508a42 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.648942 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8f59f-3f99-44f7-946f-60af9a374b33 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.654644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1770172-dacd-4a4c-a0a8-671e5cdfbbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.655357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1770172-dacd-4a4c-a0a8-671e5cdfbbe4 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.655755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1770172-dacd-4a4c-a0a8-671e5cdfbbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.656097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1770172-dacd-4a4c-a0a8-671e5cdfbbe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.657424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1770172-dacd-4a4c-a0a8-671e5cdfbbe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.659651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1770172-dacd-4a4c-a0a8-671e5cdfbbe4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.663441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:49.678154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82cbad-625b-4bea-9f0b-e52cb5c971d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.679439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82cbad-625b-4bea-9f0b-e52cb5c971d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.680020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82cbad-625b-4bea-9f0b-e52cb5c971d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.680751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82cbad-625b-4bea-9f0b-e52cb5c971d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.681215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82cbad-625b-4bea-9f0b-e52cb5c971d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.683393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82cbad-625b-4bea-9f0b-e52cb5c971d0 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.698516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 800/1584] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:49.700535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82cbad-625b-4bea-9f0b-e52cb5c971d0 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.704645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 785/1585] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.704645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1770172-dacd-4a4c-a0a8-671e5cdfbbe4 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.721580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82cbad-625b-4bea-9f0b-e52cb5c971d0 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.778537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1770172-dacd-4a4c-a0a8-671e5cdfbbe4 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.784279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2077423-260d-4a56-9b10-69f1e136ad5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.785155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2077423-260d-4a56-9b10-69f1e136ad5f admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.785988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2077423-260d-4a56-9b10-69f1e136ad5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.786344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2077423-260d-4a56-9b10-69f1e136ad5f admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.787207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2077423-260d-4a56-9b10-69f1e136ad5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.789560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2077423-260d-4a56-9b10-69f1e136ad5f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.793171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3e978-7c92-43b4-8563-6fd039a5e7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.794186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3e978-7c92-43b4-8563-6fd039a5e7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.794688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3e978-7c92-43b4-8563-6fd039a5e7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.795139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3e978-7c92-43b4-8563-6fd039a5e7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.796034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3e978-7c92-43b4-8563-6fd039a5e7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.798222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3e978-7c92-43b4-8563-6fd039a5e7f6 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.815929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 801/1586] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:49.815929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3e978-7c92-43b4-8563-6fd039a5e7f6 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.816138 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 786/1587] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.816138 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2077423-260d-4a56-9b10-69f1e136ad5f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.828616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2077423-260d-4a56-9b10-69f1e136ad5f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.830806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3e978-7c92-43b4-8563-6fd039a5e7f6 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.834048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2abc5-7493-4fea-9e2d-7e310161061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.834874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2abc5-7493-4fea-9e2d-7e310161061b admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.835269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2abc5-7493-4fea-9e2d-7e310161061b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.835657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2abc5-7493-4fea-9e2d-7e310161061b admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.836398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afa157-b2a8-4198-a936-7d606e6a3537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.836710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2abc5-7493-4fea-9e2d-7e310161061b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.837113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afa157-b2a8-4198-a936-7d606e6a3537 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.837653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afa157-b2a8-4198-a936-7d606e6a3537 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.838064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afa157-b2a8-4198-a936-7d606e6a3537 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.839198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2abc5-7493-4fea-9e2d-7e310161061b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.839564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afa157-b2a8-4198-a936-7d606e6a3537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.841226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afa157-b2a8-4198-a936-7d606e6a3537 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.859289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 802/1588] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:49.860727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afa157-b2a8-4198-a936-7d606e6a3537 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.862801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 787/1589] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/users => generated 294 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:49.867112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2abc5-7493-4fea-9e2d-7e310161061b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.871902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afa157-b2a8-4198-a936-7d606e6a3537 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.877773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9e751-be9d-449d-ba3c-58de4918732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.879075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9e751-be9d-449d-ba3c-58de4918732d admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.879581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9e751-be9d-449d-ba3c-58de4918732d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.879863 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9e751-be9d-449d-ba3c-58de4918732d admin admin] PATH_INFO: `/v3/projects/9e7b007aaf80436f8fa691a12a95f21e/users/3af6df3efbb74db4958654503e17d82e/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.880382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2abc5-7493-4fea-9e2d-7e310161061b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.886391 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd3227-8b69-4893-b4a2-37a48964a7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:49.887461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd3227-8b69-4893-b4a2-37a48964a7bb admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:49.887861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd3227-8b69-4893-b4a2-37a48964a7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:49.888249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd3227-8b69-4893-b4a2-37a48964a7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:49.889096 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd3227-8b69-4893-b4a2-37a48964a7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.891923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd3227-8b69-4893-b4a2-37a48964a7bb admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.901137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9e751-be9d-449d-ba3c-58de4918732d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7b007aaf80436f8fa691a12a95f21e, user_id=3af6df3efbb74db4958654503e17d82e, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:49.905932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9e751-be9d-449d-ba3c-58de4918732d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:49.907163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 788/1590] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:49.907163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd3227-8b69-4893-b4a2-37a48964a7bb admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.921641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd3227-8b69-4893-b4a2-37a48964a7bb admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:49.929035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 803/1591] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:49 2022] PUT /identity/v3/projects/9e7b007aaf80436f8fa691a12a95f21e/users/3af6df3efbb74db4958654503e17d82e/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:49.930438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9e751-be9d-449d-ba3c-58de4918732d admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:49.986330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:50.020190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee36257-864e-4632-a006-9e624eefcafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.021313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee36257-864e-4632-a006-9e624eefcafe admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.021737 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee36257-864e-4632-a006-9e624eefcafe admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.022420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee36257-864e-4632-a006-9e624eefcafe admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.023158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee36257-864e-4632-a006-9e624eefcafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.025085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee36257-864e-4632-a006-9e624eefcafe admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.032938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9e751-be9d-449d-ba3c-58de4918732d admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.042110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c389-8f20-4d1e-a1c5-5b3c785fcdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.042862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c389-8f20-4d1e-a1c5-5b3c785fcdfc admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.043222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c389-8f20-4d1e-a1c5-5b3c785fcdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.043576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c389-8f20-4d1e-a1c5-5b3c785fcdfc admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.044308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235c389-8f20-4d1e-a1c5-5b3c785fcdfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.045989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235c389-8f20-4d1e-a1c5-5b3c785fcdfc admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.061770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 804/1592] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:50.063508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 789/1593] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:49 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:50.063678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c389-8f20-4d1e-a1c5-5b3c785fcdfc admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.065351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee36257-864e-4632-a006-9e624eefcafe admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.077506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee36257-864e-4632-a006-9e624eefcafe admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.077688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c389-8f20-4d1e-a1c5-5b3c785fcdfc admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.083174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb788c-1f05-488e-9350-7e35a3431d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.083839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb788c-1f05-488e-9350-7e35a3431d84 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.084141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb788c-1f05-488e-9350-7e35a3431d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.084403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb788c-1f05-488e-9350-7e35a3431d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.085135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb788c-1f05-488e-9350-7e35a3431d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.086757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb788c-1f05-488e-9350-7e35a3431d84 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.087110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc7b94-6e6a-4dca-bc1b-868e530d43a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.090452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc7b94-6e6a-4dca-bc1b-868e530d43a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.090452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc7b94-6e6a-4dca-bc1b-868e530d43a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.090452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc7b94-6e6a-4dca-bc1b-868e530d43a6 admin admin] PATH_INFO: `/v3/projects/a8fb15aaa4ac4d55a68b821f99bbffd7/users/c1d2305db6284067bfb37c3b1bf198fa/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.099716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 790/1594] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:50.100579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb788c-1f05-488e-9350-7e35a3431d84 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.111571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb788c-1f05-488e-9350-7e35a3431d84 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.112345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc7b94-6e6a-4dca-bc1b-868e530d43a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8fb15aaa4ac4d55a68b821f99bbffd7, user_id=c1d2305db6284067bfb37c3b1bf198fa, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.116618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998afd1-6421-4f2c-8b54-977c3010ed66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.117292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998afd1-6421-4f2c-8b54-977c3010ed66 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.117578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998afd1-6421-4f2c-8b54-977c3010ed66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.117820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc7b94-6e6a-4dca-bc1b-868e530d43a6 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.117942 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998afd1-6421-4f2c-8b54-977c3010ed66 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.118633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998afd1-6421-4f2c-8b54-977c3010ed66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.120255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998afd1-6421-4f2c-8b54-977c3010ed66 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.137732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 791/1595] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:50.140646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998afd1-6421-4f2c-8b54-977c3010ed66 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.148483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 805/1596] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:50 2022] PUT /identity/v3/projects/a8fb15aaa4ac4d55a68b821f99bbffd7/users/c1d2305db6284067bfb37c3b1bf198fa/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:50.150362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7c45-c664-45d7-be69-fbe2b7901ffc None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.150686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7c45-c664-45d7-be69-fbe2b7901ffc None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.151069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7c45-c664-45d7-be69-fbe2b7901ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.169800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998afd1-6421-4f2c-8b54-977c3010ed66 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.187212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-24ce7c45-c664-45d7-be69-fbe2b7901ffc None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:50.189087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-24ce7c45-c664-45d7-be69-fbe2b7901ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:50.207670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:50.260532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdcabd-a632-44ae-8002-9a0ab2c9aaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.261842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdcabd-a632-44ae-8002-9a0ab2c9aaf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.262354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdcabd-a632-44ae-8002-9a0ab2c9aaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.262908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdcabd-a632-44ae-8002-9a0ab2c9aaf5 admin admin] PATH_INFO: `/v3/projects/e35635738b4746d1980b91d367546d55/users/73d38336bb17437ea02066d617d6b015/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.271059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 806/1597] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:50.272026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7c45-c664-45d7-be69-fbe2b7901ffc None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.281806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdcabd-a632-44ae-8002-9a0ab2c9aaf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35635738b4746d1980b91d367546d55, user_id=73d38336bb17437ea02066d617d6b015, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.283234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING py.warnings [None req-2acdcabd-a632-44ae-8002-9a0ab2c9aaf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:49:50.283234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: warnings.warn(msg) Jun 15 15:49:50.283234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: Jun 15 15:49:50.286559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdcabd-a632-44ae-8002-9a0ab2c9aaf5 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.291717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7c45-c664-45d7-be69-fbe2b7901ffc None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.300911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deacca-4b81-4082-a4ef-dac3cce9ee9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.301750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deacca-4b81-4082-a4ef-dac3cce9ee9b admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.302356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deacca-4b81-4082-a4ef-dac3cce9ee9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.302642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deacca-4b81-4082-a4ef-dac3cce9ee9b admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.303486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deacca-4b81-4082-a4ef-dac3cce9ee9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.305279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deacca-4b81-4082-a4ef-dac3cce9ee9b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.310784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 792/1598] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:50 2022] PUT /identity/v3/projects/e35635738b4746d1980b91d367546d55/users/73d38336bb17437ea02066d617d6b015/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:50.310784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdcabd-a632-44ae-8002-9a0ab2c9aaf5 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.330721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 807/1599] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:50.331752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deacca-4b81-4082-a4ef-dac3cce9ee9b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.407585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deacca-4b81-4082-a4ef-dac3cce9ee9b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.451384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdcabd-a632-44ae-8002-9a0ab2c9aaf5 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.461673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a3d39-a2fa-492d-9716-248f87e18a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.462935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a3d39-a2fa-492d-9716-248f87e18a78 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.463912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a3d39-a2fa-492d-9716-248f87e18a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.467657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a3d39-a2fa-492d-9716-248f87e18a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.471436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a3d39-a2fa-492d-9716-248f87e18a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.473532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a3d39-a2fa-492d-9716-248f87e18a78 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.483435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51554358-a9d4-4509-9742-4f6e7b30b14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.484992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51554358-a9d4-4509-9742-4f6e7b30b14d admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.485275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51554358-a9d4-4509-9742-4f6e7b30b14d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.485550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51554358-a9d4-4509-9742-4f6e7b30b14d admin admin] PATH_INFO: `/v3/projects/73977e69fd104c9285d2741e0bae8fa1/users/41ba7fe63355423e8438d0d1237e983a/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.493740 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 793/1600] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:50.495081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a3d39-a2fa-492d-9716-248f87e18a78 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.506546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51554358-a9d4-4509-9742-4f6e7b30b14d admin admin] RBAC: Authorizing `identity:create_grant(project_id=73977e69fd104c9285d2741e0bae8fa1, user_id=41ba7fe63355423e8438d0d1237e983a, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.509461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51554358-a9d4-4509-9742-4f6e7b30b14d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.512839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a3d39-a2fa-492d-9716-248f87e18a78 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.520689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d1dee-9d1e-4c32-9c9f-75670b8a5b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.521890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d1dee-9d1e-4c32-9c9f-75670b8a5b3a admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.522107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d1dee-9d1e-4c32-9c9f-75670b8a5b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.522461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d1dee-9d1e-4c32-9c9f-75670b8a5b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.523297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d1dee-9d1e-4c32-9c9f-75670b8a5b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.524974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d1dee-9d1e-4c32-9c9f-75670b8a5b3a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.526675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 808/1601] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:50 2022] PUT /identity/v3/projects/73977e69fd104c9285d2741e0bae8fa1/users/41ba7fe63355423e8438d0d1237e983a/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:50.528153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6c57c-2152-4fe4-b5ab-3ecffb39fee3 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.528468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6c57c-2152-4fe4-b5ab-3ecffb39fee3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.528753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6c57c-2152-4fe4-b5ab-3ecffb39fee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.548161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-cee6c57c-2152-4fe4-b5ab-3ecffb39fee3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:50.549424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cee6c57c-2152-4fe4-b5ab-3ecffb39fee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:50.553084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 794/1602] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:50.554312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d1dee-9d1e-4c32-9c9f-75670b8a5b3a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.572816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:50.620253 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 809/1603] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:50.622491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9afcb-7735-4285-a293-812d369498aa None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.622694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9afcb-7735-4285-a293-812d369498aa None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.622952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9afcb-7735-4285-a293-812d369498aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.626521 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d1dee-9d1e-4c32-9c9f-75670b8a5b3a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.653833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-c0c9afcb-7735-4285-a293-812d369498aa None None] MFA Rules not processed for user `73d38336bb17437ea02066d617d6b015`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:50.655646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c0c9afcb-7735-4285-a293-812d369498aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:50.674410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:50.690064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e8324-ecf8-4291-b639-71c0ea732cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.690808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8324-ecf8-4291-b639-71c0ea732cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.691100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8324-ecf8-4291-b639-71c0ea732cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.691520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8324-ecf8-4291-b639-71c0ea732cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.702897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e8324-ecf8-4291-b639-71c0ea732cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.704668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e8324-ecf8-4291-b639-71c0ea732cf9 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.716440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 795/1604] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:50.717559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e8324-ecf8-4291-b639-71c0ea732cf9 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.728272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 810/1605] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:50.729214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9afcb-7735-4285-a293-812d369498aa None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.735909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e8324-ecf8-4291-b639-71c0ea732cf9 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.742517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88d076-6a99-4250-b063-3a69ffe8065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.743623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88d076-6a99-4250-b063-3a69ffe8065e admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.743916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88d076-6a99-4250-b063-3a69ffe8065e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.744340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88d076-6a99-4250-b063-3a69ffe8065e admin admin] PATH_INFO: `/v3/projects/f6300459e2d1437eb386269c9e0c8a8f/users/a37287e68cdd4bc4b8ec9d20a1daec8c/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.771212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88d076-6a99-4250-b063-3a69ffe8065e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6300459e2d1437eb386269c9e0c8a8f, user_id=a37287e68cdd4bc4b8ec9d20a1daec8c, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.772438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9afcb-7735-4285-a293-812d369498aa None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.778066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88d076-6a99-4250-b063-3a69ffe8065e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.786529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6708c-c53f-451a-a08b-779ee1457823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.787818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6708c-c53f-451a-a08b-779ee1457823 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.788097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6708c-c53f-451a-a08b-779ee1457823 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.788372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6708c-c53f-451a-a08b-779ee1457823 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.789165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6708c-c53f-451a-a08b-779ee1457823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.792445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6708c-c53f-451a-a08b-779ee1457823 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.816362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 796/1606] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:50 2022] PUT /identity/v3/projects/f6300459e2d1437eb386269c9e0c8a8f/users/a37287e68cdd4bc4b8ec9d20a1daec8c/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:50.816362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96a399-a14c-444d-8ffa-c396d63d9933 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.816362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96a399-a14c-444d-8ffa-c396d63d9933 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.816362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96a399-a14c-444d-8ffa-c396d63d9933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.816810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 811/1607] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:50.816810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6708c-c53f-451a-a08b-779ee1457823 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.828136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6708c-c53f-451a-a08b-779ee1457823 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:50.859418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-6f96a399-a14c-444d-8ffa-c396d63d9933 None None] MFA Rules not processed for user `41ba7fe63355423e8438d0d1237e983a`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:50.861199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6f96a399-a14c-444d-8ffa-c396d63d9933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:50.939560 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f588f3a-3f61-4d17-acd4-a50ed887289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:50.940082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f588f3a-3f61-4d17-acd4-a50ed887289b admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.940368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f588f3a-3f61-4d17-acd4-a50ed887289b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.940648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f588f3a-3f61-4d17-acd4-a50ed887289b admin admin] PATH_INFO: `/v3/projects/c5cc94afeb35472a9f2d9cb52083da46/users/fe4516f840a2496d84091b426f056fe8/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.961550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f588f3a-3f61-4d17-acd4-a50ed887289b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5cc94afeb35472a9f2d9cb52083da46, user_id=fe4516f840a2496d84091b426f056fe8, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:50.964673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f588f3a-3f61-4d17-acd4-a50ed887289b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:50.974233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 797/1608] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:50.974830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96a399-a14c-444d-8ffa-c396d63d9933 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:50.989479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 812/1609] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:50 2022] PUT /identity/v3/projects/c5cc94afeb35472a9f2d9cb52083da46/users/fe4516f840a2496d84091b426f056fe8/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:50.991276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb766f-2735-47d8-bf24-66b93bb9ad5e None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.991616 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb766f-2735-47d8-bf24-66b93bb9ad5e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.991903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb766f-2735-47d8-bf24-66b93bb9ad5e None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:50.993649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 813/1610] 10.222.0.62 () {54 vars in 961 bytes} [Wed Jun 15 15:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:49:50.994898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517e0d3-69d6-48d1-95f2-f1303354fce9 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:50.995201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517e0d3-69d6-48d1-95f2-f1303354fce9 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:50.995481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517e0d3-69d6-48d1-95f2-f1303354fce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.053680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:51.060012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4517e0d3-69d6-48d1-95f2-f1303354fce9 None None] MFA Rules not processed for user `a37287e68cdd4bc4b8ec9d20a1daec8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:51.061705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4517e0d3-69d6-48d1-95f2-f1303354fce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:51.075523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96a399-a14c-444d-8ffa-c396d63d9933 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:51.180224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2651896-30dd-4782-8dce-f85a6c6d7f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:51.180893 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2651896-30dd-4782-8dce-f85a6c6d7f8c service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:51.181182 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2651896-30dd-4782-8dce-f85a6c6d7f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:51.181619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2651896-30dd-4782-8dce-f85a6c6d7f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.213189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 814/1611] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:51.214115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517e0d3-69d6-48d1-95f2-f1303354fce9 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:51.258369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:51.310488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2651896-30dd-4782-8dce-f85a6c6d7f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:51.312576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2651896-30dd-4782-8dce-f85a6c6d7f8c service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:51.326395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 798/1612] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:51.330264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2651896-30dd-4782-8dce-f85a6c6d7f8c service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:51.350672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517e0d3-69d6-48d1-95f2-f1303354fce9 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:51.352573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2651896-30dd-4782-8dce-f85a6c6d7f8c service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:51.361176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c3c00-3109-4237-9e7a-099dbeaa0f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:51.363515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c3c00-3109-4237-9e7a-099dbeaa0f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:51.364222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d016be-8938-4d9e-bb04-2882f82fd3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:51.365114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d016be-8938-4d9e-bb04-2882f82fd3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:51.365414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d016be-8938-4d9e-bb04-2882f82fd3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:51.365747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d016be-8938-4d9e-bb04-2882f82fd3e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.366714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d016be-8938-4d9e-bb04-2882f82fd3e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:51.367151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c3c00-3109-4237-9e7a-099dbeaa0f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:51.368508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c3c00-3109-4237-9e7a-099dbeaa0f26 admin admin] PATH_INFO: `/v3/projects/0496c4ee9cdf486684000904876d0b6b/users/fb838f931b7748e2baf6c2315d3cbd75/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.368876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d016be-8938-4d9e-bb04-2882f82fd3e4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:51.404763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c3c00-3109-4237-9e7a-099dbeaa0f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0496c4ee9cdf486684000904876d0b6b, user_id=fb838f931b7748e2baf6c2315d3cbd75, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:51.412572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c3c00-3109-4237-9e7a-099dbeaa0f26 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:51.419030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 799/1613] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:51 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:51.421511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180e1d-398b-405c-8158-4092e1d1c712 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:51.421634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180e1d-398b-405c-8158-4092e1d1c712 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:51.422359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180e1d-398b-405c-8158-4092e1d1c712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.447852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 815/1614] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:51 2022] PUT /identity/v3/projects/0496c4ee9cdf486684000904876d0b6b/users/fb838f931b7748e2baf6c2315d3cbd75/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:51.449245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77428e08-754a-40a7-b870-af9b01d0d36d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:51.449592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77428e08-754a-40a7-b870-af9b01d0d36d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:51.449877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77428e08-754a-40a7-b870-af9b01d0d36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.451192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-dd180e1d-398b-405c-8158-4092e1d1c712 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:51.453802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-dd180e1d-398b-405c-8158-4092e1d1c712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:51.491890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-77428e08-754a-40a7-b870-af9b01d0d36d None None] MFA Rules not processed for user `fe4516f840a2496d84091b426f056fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:51.495199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-77428e08-754a-40a7-b870-af9b01d0d36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:51.582754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 800/1615] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Jun 15 15:49:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:51.583481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd180e1d-398b-405c-8158-4092e1d1c712 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:51.614089 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:51.664695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 816/1616] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:51.670940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77428e08-754a-40a7-b870-af9b01d0d36d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:51.731507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd180e1d-398b-405c-8158-4092e1d1c712 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:51.738905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe5e3a-d6fe-4601-be55-9ae9f7faf0d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:51.739145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe5e3a-d6fe-4601-be55-9ae9f7faf0d6 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:51.739500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe5e3a-d6fe-4601-be55-9ae9f7faf0d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:51.739880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe5e3a-d6fe-4601-be55-9ae9f7faf0d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.824314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77428e08-754a-40a7-b870-af9b01d0d36d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:51.837205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda486-eb86-4526-a3e6-3fb37b04da95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:51.840875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda486-eb86-4526-a3e6-3fb37b04da95 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:51.840875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda486-eb86-4526-a3e6-3fb37b04da95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:51.840875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda486-eb86-4526-a3e6-3fb37b04da95 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.843375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda486-eb86-4526-a3e6-3fb37b04da95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:51.850122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda486-eb86-4526-a3e6-3fb37b04da95 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:51.872592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe5e3a-d6fe-4601-be55-9ae9f7faf0d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:51.875906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe5e3a-d6fe-4601-be55-9ae9f7faf0d6 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:51.912027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 817/1617] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:51 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:51.916339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 801/1618] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:51.917580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe5e3a-d6fe-4601-be55-9ae9f7faf0d6 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:51.917969 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0768b6-ae4d-42ef-bad6-420ed0003f0a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:51.919371 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0768b6-ae4d-42ef-bad6-420ed0003f0a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:51.919908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0768b6-ae4d-42ef-bad6-420ed0003f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.957598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe5e3a-d6fe-4601-be55-9ae9f7faf0d6 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:51.973048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d42aed-7b47-4d05-8fcc-dc87d579423a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:51.974916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d42aed-7b47-4d05-8fcc-dc87d579423a service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:51.975265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ca0768b6-ae4d-42ef-bad6-420ed0003f0a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:51.975652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d42aed-7b47-4d05-8fcc-dc87d579423a service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:51.976460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d42aed-7b47-4d05-8fcc-dc87d579423a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:51.979173 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ca0768b6-ae4d-42ef-bad6-420ed0003f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:52.001391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 818/1619] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:52.002174 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0768b6-ae4d-42ef-bad6-420ed0003f0a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.023111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0768b6-ae4d-42ef-bad6-420ed0003f0a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.037186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9585c5f-95bb-495d-90c7-389d1a83b84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.038259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9585c5f-95bb-495d-90c7-389d1a83b84a admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.038259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9585c5f-95bb-495d-90c7-389d1a83b84a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.038872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9585c5f-95bb-495d-90c7-389d1a83b84a admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.039725 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9585c5f-95bb-495d-90c7-389d1a83b84a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.041755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9585c5f-95bb-495d-90c7-389d1a83b84a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.076255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:52.101095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 819/1620] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:52 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:52.105781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9585c5f-95bb-495d-90c7-389d1a83b84a admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.124437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d42aed-7b47-4d05-8fcc-dc87d579423a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.129747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d42aed-7b47-4d05-8fcc-dc87d579423a service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.161695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 802/1621] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:52.164691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d42aed-7b47-4d05-8fcc-dc87d579423a service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.287374 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9585c5f-95bb-495d-90c7-389d1a83b84a admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.299467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d43566-7ce0-4a66-9afa-725d9e99b677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.301659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d43566-7ce0-4a66-9afa-725d9e99b677 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.302031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d43566-7ce0-4a66-9afa-725d9e99b677 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.302552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d43566-7ce0-4a66-9afa-725d9e99b677 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.304136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d43566-7ce0-4a66-9afa-725d9e99b677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.308039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d43566-7ce0-4a66-9afa-725d9e99b677 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.341249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 820/1622] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:52.341249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d43566-7ce0-4a66-9afa-725d9e99b677 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.367802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d43566-7ce0-4a66-9afa-725d9e99b677 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.388485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabd254-dd84-4210-b083-c1b20b66442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.389053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabd254-dd84-4210-b083-c1b20b66442b admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.389291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabd254-dd84-4210-b083-c1b20b66442b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.389571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabd254-dd84-4210-b083-c1b20b66442b admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.390326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabd254-dd84-4210-b083-c1b20b66442b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.392353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabd254-dd84-4210-b083-c1b20b66442b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.443898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 821/1623] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:52 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:52.445308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabd254-dd84-4210-b083-c1b20b66442b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.472229 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabd254-dd84-4210-b083-c1b20b66442b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.489467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9b03-3966-4083-8b06-19d64450ad3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.491356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9b03-3966-4083-8b06-19d64450ad3d admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.491631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9b03-3966-4083-8b06-19d64450ad3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.491910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9b03-3966-4083-8b06-19d64450ad3d admin admin] PATH_INFO: `/v3/projects/6b39759a4340420886bd14c137162e29/users/94cffdd804e448dbb91a375addfa01d3/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.531850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d9b03-3966-4083-8b06-19d64450ad3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b39759a4340420886bd14c137162e29, user_id=94cffdd804e448dbb91a375addfa01d3, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.535729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d9b03-3966-4083-8b06-19d64450ad3d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.573183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 822/1624] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:52 2022] PUT /identity/v3/projects/6b39759a4340420886bd14c137162e29/users/94cffdd804e448dbb91a375addfa01d3/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:52.573183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9b03-3966-4083-8b06-19d64450ad3d admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.621664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d42aed-7b47-4d05-8fcc-dc87d579423a service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.671132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:52.723039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9b03-3966-4083-8b06-19d64450ad3d admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.741869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78671ab7-18aa-4748-a5ca-13f4f1afdb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.744127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78671ab7-18aa-4748-a5ca-13f4f1afdb6a admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.746618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78671ab7-18aa-4748-a5ca-13f4f1afdb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.747008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78671ab7-18aa-4748-a5ca-13f4f1afdb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.748674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78671ab7-18aa-4748-a5ca-13f4f1afdb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.752395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78671ab7-18aa-4748-a5ca-13f4f1afdb6a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.760818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e1027-84ae-4eb6-acd9-158d0c8764dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.763466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e1027-84ae-4eb6-acd9-158d0c8764dd service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.763851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e1027-84ae-4eb6-acd9-158d0c8764dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.764596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e1027-84ae-4eb6-acd9-158d0c8764dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.783058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 823/1625] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:52.784181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78671ab7-18aa-4748-a5ca-13f4f1afdb6a admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.795502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e1027-84ae-4eb6-acd9-158d0c8764dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.801761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e1027-84ae-4eb6-acd9-158d0c8764dd service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.808282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78671ab7-18aa-4748-a5ca-13f4f1afdb6a admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.821177 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282f10d-d69a-42e0-b130-0cb32788c04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.821870 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282f10d-d69a-42e0-b130-0cb32788c04c admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.822150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282f10d-d69a-42e0-b130-0cb32788c04c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.822502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282f10d-d69a-42e0-b130-0cb32788c04c admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.824736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282f10d-d69a-42e0-b130-0cb32788c04c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.826246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 803/1626] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:52.829204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282f10d-d69a-42e0-b130-0cb32788c04c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.831662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e1027-84ae-4eb6-acd9-158d0c8764dd service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.859853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e1027-84ae-4eb6-acd9-158d0c8764dd service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.861486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 824/1627] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:52.871262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282f10d-d69a-42e0-b130-0cb32788c04c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.877327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4485165-c42e-497e-8c8f-547c8ae3ece2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.878435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4485165-c42e-497e-8c8f-547c8ae3ece2 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.879623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4485165-c42e-497e-8c8f-547c8ae3ece2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.879987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4485165-c42e-497e-8c8f-547c8ae3ece2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.881416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4485165-c42e-497e-8c8f-547c8ae3ece2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.885978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4485165-c42e-497e-8c8f-547c8ae3ece2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.898960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282f10d-d69a-42e0-b130-0cb32788c04c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.902771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 804/1628] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:52.907092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4485165-c42e-497e-8c8f-547c8ae3ece2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:52.914009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8d355-aeab-4b9f-a44c-d767f9eb2050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.914949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8d355-aeab-4b9f-a44c-d767f9eb2050 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.915233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8d355-aeab-4b9f-a44c-d767f9eb2050 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.915514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8d355-aeab-4b9f-a44c-d767f9eb2050 admin admin] PATH_INFO: `/v3/projects/6b39759a4340420886bd14c137162e29/users/94cffdd804e448dbb91a375addfa01d3/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.920045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8d355-aeab-4b9f-a44c-d767f9eb2050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b39759a4340420886bd14c137162e29, user_id=94cffdd804e448dbb91a375addfa01d3, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.921739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4485165-c42e-497e-8c8f-547c8ae3ece2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:52.923355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8d355-aeab-4b9f-a44c-d767f9eb2050 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.928682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4399a-eccb-4253-ab94-aaf4ce151d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:52.930720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4399a-eccb-4253-ab94-aaf4ce151d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.931280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4399a-eccb-4253-ab94-aaf4ce151d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.931832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4399a-eccb-4253-ab94-aaf4ce151d10 admin admin] PATH_INFO: `/v3/projects/53685cedcf4c429f8405362f51716015/users/9af376ff010644e7b5b59bb9abbaef82/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:52.952589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4399a-eccb-4253-ab94-aaf4ce151d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53685cedcf4c429f8405362f51716015, user_id=9af376ff010644e7b5b59bb9abbaef82, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:52.956839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4399a-eccb-4253-ab94-aaf4ce151d10 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:52.964166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 825/1629] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:52 2022] PUT /identity/v3/projects/6b39759a4340420886bd14c137162e29/users/94cffdd804e448dbb91a375addfa01d3/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:52.965807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89918aa-e8ac-452f-aafe-e009585c19f4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:52.966187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89918aa-e8ac-452f-aafe-e009585c19f4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:52.966469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89918aa-e8ac-452f-aafe-e009585c19f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.003277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 805/1630] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:52 2022] PUT /identity/v3/projects/53685cedcf4c429f8405362f51716015/users/9af376ff010644e7b5b59bb9abbaef82/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:49:53.005446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3588d-5f19-40dd-88b2-c8a2f39012b0 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.005972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3588d-5f19-40dd-88b2-c8a2f39012b0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.006593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3588d-5f19-40dd-88b2-c8a2f39012b0 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.008787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 806/1631] 10.222.0.62 () {54 vars in 912 bytes} [Wed Jun 15 15:49:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:49:53.009767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e89918aa-e8ac-452f-aafe-e009585c19f4 None None] MFA Rules not processed for user `c1d2305db6284067bfb37c3b1bf198fa`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:53.010748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602f599-2bbd-457a-b30b-1ca0e4f6158f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.011198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e89918aa-e8ac-452f-aafe-e009585c19f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:53.011480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602f599-2bbd-457a-b30b-1ca0e4f6158f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.011974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602f599-2bbd-457a-b30b-1ca0e4f6158f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.041521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:53.048186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-5602f599-2bbd-457a-b30b-1ca0e4f6158f None None] MFA Rules not processed for user `94cffdd804e448dbb91a375addfa01d3`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:53.049875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5602f599-2bbd-457a-b30b-1ca0e4f6158f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:53.166135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 807/1632] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:53.170612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018a316-5c42-4be3-a370-b84f8f1b10d4 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.171268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018a316-5c42-4be3-a370-b84f8f1b10d4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.171815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018a316-5c42-4be3-a370-b84f8f1b10d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.191867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 826/1633] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:53.193831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbf51b-308a-46bc-9128-024b45c9de37 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.194005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbf51b-308a-46bc-9128-024b45c9de37 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.194268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbf51b-308a-46bc-9128-024b45c9de37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.218438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-1018a316-5c42-4be3-a370-b84f8f1b10d4 None None] MFA Rules not processed for user `9af376ff010644e7b5b59bb9abbaef82`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:53.221616 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6ccbf51b-308a-46bc-9128-024b45c9de37 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:53.222523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1018a316-5c42-4be3-a370-b84f8f1b10d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:53.223205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6ccbf51b-308a-46bc-9128-024b45c9de37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:53.361919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 808/1634] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:53.363289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018a316-5c42-4be3-a370-b84f8f1b10d4 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:53.369979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:53.418584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 827/1635] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:53.440605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069d0eb-85e6-4ec3-99e4-33f1bbe6e430 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.442991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069d0eb-85e6-4ec3-99e4-33f1bbe6e430 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.443450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069d0eb-85e6-4ec3-99e4-33f1bbe6e430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.497093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018a316-5c42-4be3-a370-b84f8f1b10d4 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:53.498374 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-c069d0eb-85e6-4ec3-99e4-33f1bbe6e430 None None] MFA Rules not processed for user `c1d2305db6284067bfb37c3b1bf198fa`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:53.506326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c069d0eb-85e6-4ec3-99e4-33f1bbe6e430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:53.510916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc0ec9-fa74-42be-9485-2f38b7f0277d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:53.511660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc0ec9-fa74-42be-9485-2f38b7f0277d service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.512176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc0ec9-fa74-42be-9485-2f38b7f0277d service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.512338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc0ec9-fa74-42be-9485-2f38b7f0277d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.532155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 828/1636] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:53.534654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc0ec9-fa74-42be-9485-2f38b7f0277d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:53.536075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc0ec9-fa74-42be-9485-2f38b7f0277d service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:53.536220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069d0eb-85e6-4ec3-99e4-33f1bbe6e430 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:53.575136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 809/1637] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:53.577337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc0ec9-fa74-42be-9485-2f38b7f0277d service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:53.598075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc0ec9-fa74-42be-9485-2f38b7f0277d service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:53.610018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbccd-4c03-4673-a04e-b41241e53b6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:53.611447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbccd-4c03-4673-a04e-b41241e53b6d service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.611733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbccd-4c03-4673-a04e-b41241e53b6d service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.612329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbccd-4c03-4673-a04e-b41241e53b6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.639343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dbccd-4c03-4673-a04e-b41241e53b6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:53.647797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dbccd-4c03-4673-a04e-b41241e53b6d service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:53.698865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 810/1638] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:53.707672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a8612-59e4-4274-9836-a7b374b06467 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.712735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a8612-59e4-4274-9836-a7b374b06467 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.713085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a8612-59e4-4274-9836-a7b374b06467 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.721207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 811/1639] 10.222.0.62 () {54 vars in 961 bytes} [Wed Jun 15 15:49:53 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:49:53.725041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf0338-e2de-4cdf-b0b2-86509eb43780 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.725339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf0338-e2de-4cdf-b0b2-86509eb43780 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.725778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf0338-e2de-4cdf-b0b2-86509eb43780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.785970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-07bf0338-e2de-4cdf-b0b2-86509eb43780 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:53.798929 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-07bf0338-e2de-4cdf-b0b2-86509eb43780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:53.843424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 812/1640] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Jun 15 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:53.846881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf0338-e2de-4cdf-b0b2-86509eb43780 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:53.860335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069d0eb-85e6-4ec3-99e4-33f1bbe6e430 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:53.869523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf0338-e2de-4cdf-b0b2-86509eb43780 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:53.880295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c43290-2510-4ab6-b6c9-5f49baadecc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:53.880989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c43290-2510-4ab6-b6c9-5f49baadecc0 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.881272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c43290-2510-4ab6-b6c9-5f49baadecc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.881549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c43290-2510-4ab6-b6c9-5f49baadecc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.884572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dff4c6-3e0e-4581-a240-47e911760dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:53.885340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dff4c6-3e0e-4581-a240-47e911760dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:53.885523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dff4c6-3e0e-4581-a240-47e911760dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:53.885814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dff4c6-3e0e-4581-a240-47e911760dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:53.886876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dff4c6-3e0e-4581-a240-47e911760dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:53.888818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dff4c6-3e0e-4581-a240-47e911760dc7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:53.976312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 829/1641] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:49:53 2022] POST /identity/v3/projects => generated 383 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:53.977626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dff4c6-3e0e-4581-a240-47e911760dc7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:54.010793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:54.027445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c43290-2510-4ab6-b6c9-5f49baadecc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:54.033482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c43290-2510-4ab6-b6c9-5f49baadecc0 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:54.076110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 813/1642] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:54.081219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c43290-2510-4ab6-b6c9-5f49baadecc0 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:54.118450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c43290-2510-4ab6-b6c9-5f49baadecc0 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:54.149158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dff4c6-3e0e-4581-a240-47e911760dc7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:54.168918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369fef-7f45-4c87-b702-2ae85770aac3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:54.169907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369fef-7f45-4c87-b702-2ae85770aac3 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:54.170365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369fef-7f45-4c87-b702-2ae85770aac3 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:54.170856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369fef-7f45-4c87-b702-2ae85770aac3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:54.284843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d34d3-46c3-4a83-8617-9b6c28ae1b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:54.287112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d34d3-46c3-4a83-8617-9b6c28ae1b0d admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:54.287398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d34d3-46c3-4a83-8617-9b6c28ae1b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:54.287678 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d34d3-46c3-4a83-8617-9b6c28ae1b0d admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:54.290786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d34d3-46c3-4a83-8617-9b6c28ae1b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:54.297019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d34d3-46c3-4a83-8617-9b6c28ae1b0d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:54.317364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369fef-7f45-4c87-b702-2ae85770aac3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:54.320685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369fef-7f45-4c87-b702-2ae85770aac3 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:54.340692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 830/1643] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:54.360149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 814/1644] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:49:54 2022] POST /identity/v3/users => generated 298 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:49:54.361447 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369fef-7f45-4c87-b702-2ae85770aac3 service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:54.391709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369fef-7f45-4c87-b702-2ae85770aac3 service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:54.404950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08190c3a-817c-4039-81eb-ebc9229e0774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:54.405649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08190c3a-817c-4039-81eb-ebc9229e0774 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:54.405939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08190c3a-817c-4039-81eb-ebc9229e0774 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:54.407735 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08190c3a-817c-4039-81eb-ebc9229e0774 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:54.408561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08190c3a-817c-4039-81eb-ebc9229e0774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:54.413614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08190c3a-817c-4039-81eb-ebc9229e0774 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:54.459693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 831/1645] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:49:54.464585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d34d3-46c3-4a83-8617-9b6c28ae1b0d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:54.483677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d34d3-46c3-4a83-8617-9b6c28ae1b0d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:54.509491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ecf84-9791-42cf-aa52-83394fd90fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:54.512200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ecf84-9791-42cf-aa52-83394fd90fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:54.513570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ecf84-9791-42cf-aa52-83394fd90fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:54.515519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ecf84-9791-42cf-aa52-83394fd90fbf admin admin] PATH_INFO: `/v3/projects/02af1123fc984294901d0d741f64b37a/users/eedfa25f9e1249a8b13db83c5bc9c0f8/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:54.564738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:54.576519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ecf84-9791-42cf-aa52-83394fd90fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=02af1123fc984294901d0d741f64b37a, user_id=eedfa25f9e1249a8b13db83c5bc9c0f8, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:54.587996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ecf84-9791-42cf-aa52-83394fd90fbf admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:54.638109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 815/1646] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:54 2022] PUT /identity/v3/projects/02af1123fc984294901d0d741f64b37a/users/eedfa25f9e1249a8b13db83c5bc9c0f8/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:54.642601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08190c3a-817c-4039-81eb-ebc9229e0774 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:54.665265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08190c3a-817c-4039-81eb-ebc9229e0774 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:54.852905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa92c8-4c3c-41f0-bf14-ceaa19f30072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:54.853822 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa92c8-4c3c-41f0-bf14-ceaa19f30072 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:54.854220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa92c8-4c3c-41f0-bf14-ceaa19f30072 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:54.854925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa92c8-4c3c-41f0-bf14-ceaa19f30072 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:54.855939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa92c8-4c3c-41f0-bf14-ceaa19f30072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:54.858442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa92c8-4c3c-41f0-bf14-ceaa19f30072 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:54.888198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 832/1647] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:49:54.892914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ecf84-9791-42cf-aa52-83394fd90fbf admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:54.900197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858ea53-5c42-4a91-b0f7-dac4149771e8 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:54.900598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858ea53-5c42-4a91-b0f7-dac4149771e8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:54.900991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858ea53-5c42-4a91-b0f7-dac4149771e8 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:54.906922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 833/1648] 10.222.0.62 () {54 vars in 912 bytes} [Wed Jun 15 15:49:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:49:54.924356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ecf84-9791-42cf-aa52-83394fd90fbf admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:54.927888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b36ce-0232-4804-8cc4-92441abec960 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:54.928182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b36ce-0232-4804-8cc4-92441abec960 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:54.928430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b36ce-0232-4804-8cc4-92441abec960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:54.952279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09940d6-c4c6-440d-a897-d3437f2f926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:54.953918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09940d6-c4c6-440d-a897-d3437f2f926c admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:54.954298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09940d6-c4c6-440d-a897-d3437f2f926c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:54.954591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09940d6-c4c6-440d-a897-d3437f2f926c admin admin] PATH_INFO: `/v3/projects/02af1123fc984294901d0d741f64b37a/users/eedfa25f9e1249a8b13db83c5bc9c0f8/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:54.965195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09940d6-c4c6-440d-a897-d3437f2f926c admin admin] RBAC: Authorizing `identity:create_grant(project_id=02af1123fc984294901d0d741f64b37a, user_id=eedfa25f9e1249a8b13db83c5bc9c0f8, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:54.968074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09940d6-c4c6-440d-a897-d3437f2f926c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:54.969883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-446b36ce-0232-4804-8cc4-92441abec960 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:54.971404 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-446b36ce-0232-4804-8cc4-92441abec960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:54.994301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 816/1649] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:49:54 2022] PUT /identity/v3/projects/02af1123fc984294901d0d741f64b37a/users/eedfa25f9e1249a8b13db83c5bc9c0f8/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:55.005833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ccdb4-be37-49b2-b6ec-bc33c301cfad None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:55.006296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ccdb4-be37-49b2-b6ec-bc33c301cfad None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:55.006676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ccdb4-be37-49b2-b6ec-bc33c301cfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:55.079441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f89ccdb4-be37-49b2-b6ec-bc33c301cfad None None] MFA Rules not processed for user `eedfa25f9e1249a8b13db83c5bc9c0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:55.080821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f89ccdb4-be37-49b2-b6ec-bc33c301cfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:55.099213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:55.184465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 834/1650] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:49:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:55.210890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b36ce-0232-4804-8cc4-92441abec960 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:55.225454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-446b36ce-0232-4804-8cc4-92441abec960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:55.258785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 817/1651] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:55.368866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ccdb4-be37-49b2-b6ec-bc33c301cfad None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:55.411429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b36ce-0232-4804-8cc4-92441abec960 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:55.503592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ccdb4-be37-49b2-b6ec-bc33c301cfad None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:55.519992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5725c01-b5c1-485d-99e6-340fd224a472 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:55.521521 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f877d-4299-4fea-9eb4-31ad390c54ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:55.521666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5725c01-b5c1-485d-99e6-340fd224a472 service glance] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:55.522365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5725c01-b5c1-485d-99e6-340fd224a472 service glance] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:55.522497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f877d-4299-4fea-9eb4-31ad390c54ee service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:55.522606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5725c01-b5c1-485d-99e6-340fd224a472 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:55.522697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f877d-4299-4fea-9eb4-31ad390c54ee service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:55.522877 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f877d-4299-4fea-9eb4-31ad390c54ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:55.548648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f877d-4299-4fea-9eb4-31ad390c54ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:55.550484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f877d-4299-4fea-9eb4-31ad390c54ee service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:55.575679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 818/1652] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:49:55.740743 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5725c01-b5c1-485d-99e6-340fd224a472 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:55.748130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5725c01-b5c1-485d-99e6-340fd224a472 service glance] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:55.815171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 835/1653] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:56.710303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20b71-d942-4188-ab99-f69461023b0d None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:56.710785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20b71-d942-4188-ab99-f69461023b0d None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:56.710901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20b71-d942-4188-ab99-f69461023b0d None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:56.712579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 819/1654] 10.222.0.62 () {54 vars in 912 bytes} [Wed Jun 15 15:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:49:56.719528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae6240-619c-4335-91a7-12b6c5a7ef4b None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:56.720458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae6240-619c-4335-91a7-12b6c5a7ef4b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:56.720935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae6240-619c-4335-91a7-12b6c5a7ef4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:56.804456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-7eae6240-619c-4335-91a7-12b6c5a7ef4b None None] MFA Rules not processed for user `bf1915b9919442d692bcedf25f53320c`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:56.807218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7eae6240-619c-4335-91a7-12b6c5a7ef4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:56.829796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:56.908502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 836/1655] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:49:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:56.920477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede20b71-d942-4188-ab99-f69461023b0d None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:56.933170 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede20b71-d942-4188-ab99-f69461023b0d None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:56.937354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7348ee6-9371-4979-b8f2-6a8d60472a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:56.938287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7348ee6-9371-4979-b8f2-6a8d60472a05 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:56.938675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7348ee6-9371-4979-b8f2-6a8d60472a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:56.939362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7348ee6-9371-4979-b8f2-6a8d60472a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:56.956619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7348ee6-9371-4979-b8f2-6a8d60472a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:56.958063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7348ee6-9371-4979-b8f2-6a8d60472a05 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:56.967825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 820/1656] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:57.348206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae6240-619c-4335-91a7-12b6c5a7ef4b None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:57.365304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae6240-619c-4335-91a7-12b6c5a7ef4b None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:57.374136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c71b66-9a4c-4393-a07b-6bba96161707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:57.380785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71b66-9a4c-4393-a07b-6bba96161707 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:57.381226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71b66-9a4c-4393-a07b-6bba96161707 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:57.381669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71b66-9a4c-4393-a07b-6bba96161707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:57.552645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c71b66-9a4c-4393-a07b-6bba96161707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:57.554114 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c71b66-9a4c-4393-a07b-6bba96161707 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:57.578517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 837/1657] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:57.986666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224dacd-2d3c-4f4d-b34c-691d61174cdf None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:57.987658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224dacd-2d3c-4f4d-b34c-691d61174cdf None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:57.988381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224dacd-2d3c-4f4d-b34c-691d61174cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:58.101963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ce302-3c82-42a9-8a81-159e1e8090a3 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:58.102604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ce302-3c82-42a9-8a81-159e1e8090a3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:58.103266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ce302-3c82-42a9-8a81-159e1e8090a3 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:58.104980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 838/1658] 10.222.0.62 () {54 vars in 960 bytes} [Wed Jun 15 15:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:49:58.106683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f224dacd-2d3c-4f4d-b34c-691d61174cdf None None] MFA Rules not processed for user `434c81b82dee404f8d143f1bdf56d3ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:58.112820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff47c2-a9fd-4f62-be07-6a81097528d9 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:58.113200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff47c2-a9fd-4f62-be07-6a81097528d9 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:58.114158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff47c2-a9fd-4f62-be07-6a81097528d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:58.128204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f224dacd-2d3c-4f4d-b34c-691d61174cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:58.173578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e8ff47c2-a9fd-4f62-be07-6a81097528d9 None None] MFA Rules not processed for user `a786a95c48c348dbba269e0e33265271`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:58.176493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e8ff47c2-a9fd-4f62-be07-6a81097528d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:58.289627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:58.384549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 821/1659] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:58.385774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224dacd-2d3c-4f4d-b34c-691d61174cdf None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:58.408433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 839/1660] 10.222.0.62 () {58 vars in 1057 bytes} [Wed Jun 15 15:49:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:49:58.409635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff47c2-a9fd-4f62-be07-6a81097528d9 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:58.434963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff47c2-a9fd-4f62-be07-6a81097528d9 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:58.455776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864256d-cb14-43fd-ae60-fff85121968f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:58.457608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864256d-cb14-43fd-ae60-fff85121968f service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:58.458133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864256d-cb14-43fd-ae60-fff85121968f service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:58.458619 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864256d-cb14-43fd-ae60-fff85121968f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:58.481198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864256d-cb14-43fd-ae60-fff85121968f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:58.486747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864256d-cb14-43fd-ae60-fff85121968f service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:58.508852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 840/1661] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:58.510073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864256d-cb14-43fd-ae60-fff85121968f service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:58.534190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864256d-cb14-43fd-ae60-fff85121968f service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:58.545432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224dacd-2d3c-4f4d-b34c-691d61174cdf None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:58.546523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37453f-f2fa-4a22-8a04-87b82a3c1963 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:58.548780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37453f-f2fa-4a22-8a04-87b82a3c1963 service placement] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:58.549775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37453f-f2fa-4a22-8a04-87b82a3c1963 service placement] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:58.550717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37453f-f2fa-4a22-8a04-87b82a3c1963 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:58.556223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fd37453f-f2fa-4a22-8a04-87b82a3c1963 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:58.558473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:58.560296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:58.560879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:58.564012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] PATH_INFO: `/v3/users/73d38336bb17437ea02066d617d6b015` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:58.566629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d38336bb17437ea02066d617d6b015)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:58.569635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: WARNING py.warnings [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:49:58.569635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: warnings.warn(msg) Jun 15 15:49:58.569635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: Jun 15 15:49:58.571654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:58.584556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37453f-f2fa-4a22-8a04-87b82a3c1963 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:58.587206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37453f-f2fa-4a22-8a04-87b82a3c1963 service placement] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:58.611778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 841/1662] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:49:58.702649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d38336bb17437ea02066d617d6b015'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:49:58.721527 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d38336bb17437ea02066d617d6b015'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:49:58.756234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d38336bb17437ea02066d617d6b015'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:49:58.759372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d38336bb17437ea02066d617d6b015'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:49:58.761656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d38336bb17437ea02066d617d6b015'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:49:58.860029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 822/1663] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:49:58 2022] DELETE /identity/v3/users/73d38336bb17437ea02066d617d6b015 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:49:58.864932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71582e10-13f5-4fd1-92dd-f53c5c98caf9 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:58.865608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71582e10-13f5-4fd1-92dd-f53c5c98caf9 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:58.866105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71582e10-13f5-4fd1-92dd-f53c5c98caf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:58.889915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-71582e10-13f5-4fd1-92dd-f53c5c98caf9 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:49:58.892153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-71582e10-13f5-4fd1-92dd-f53c5c98caf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:58.931329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:49:58.972607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 842/1664] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:49:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:49:59.081310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:49:59.084166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:49:59.242748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f9710-d1bd-4587-bd66-95407cf83cc6 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:49:59.247864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a331d7a-cea7-4744-bf6f-d6e4522a77ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:49:59.248756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a331d7a-cea7-4744-bf6f-d6e4522a77ce service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:49:59.249190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a331d7a-cea7-4744-bf6f-d6e4522a77ce service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:49:59.249617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a331d7a-cea7-4744-bf6f-d6e4522a77ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:49:59.271590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a331d7a-cea7-4744-bf6f-d6e4522a77ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:49:59.273092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a331d7a-cea7-4744-bf6f-d6e4522a77ce service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:49:59.308142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 823/1665] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:01.328568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71582e10-13f5-4fd1-92dd-f53c5c98caf9 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:01.333791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-71582e10-13f5-4fd1-92dd-f53c5c98caf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:01.352759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71582e10-13f5-4fd1-92dd-f53c5c98caf9 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:01.371478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:01.372419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:01.372647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:01.372922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] PATH_INFO: `/v3/projects/e35635738b4746d1980b91d367546d55` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:01.374158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35635738b4746d1980b91d367546d55)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:01.375306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: WARNING py.warnings [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 15:50:01.375306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: warnings.warn(msg) Jun 15 15:50:01.375306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: Jun 15 15:50:01.376490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:01.544583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35635738b4746d1980b91d367546d55', 'request_id': 'req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:01.603659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35635738b4746d1980b91d367546d55', 'request_id': 'req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:01.636203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35635738b4746d1980b91d367546d55', 'request_id': 'req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:01.682097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] The token cache is being invalidate because project e35635738b4746d1980b91d367546d55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:01.682810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:01.683828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f1d57339-32bb-4cc6-b5ac-915d8e6923e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:01.702768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 843/1666] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:01 2022] DELETE /identity/v3/projects/e35635738b4746d1980b91d367546d55 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:01.707589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd218d43-29b4-4078-bc92-fcc145ffdc52 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:01.708122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd218d43-29b4-4078-bc92-fcc145ffdc52 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:01.708201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd218d43-29b4-4078-bc92-fcc145ffdc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:01.720079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:01.750401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-bd218d43-29b4-4078-bc92-fcc145ffdc52 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:01.751669 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-bd218d43-29b4-4078-bc92-fcc145ffdc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:01.954362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 824/1667] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:01.961990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9aca9-368b-49d6-ad69-e872a939f8f8 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:01.962521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9aca9-368b-49d6-ad69-e872a939f8f8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:01.962937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9aca9-368b-49d6-ad69-e872a939f8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:02.011892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2bb9aca9-368b-49d6-ad69-e872a939f8f8 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:02.016794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2bb9aca9-368b-49d6-ad69-e872a939f8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:02.068017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 844/1668] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:02.086038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd218d43-29b4-4078-bc92-fcc145ffdc52 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:02.105404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd218d43-29b4-4078-bc92-fcc145ffdc52 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:02.111500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48374e6d-e275-4287-bc95-f4543cf77256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:02.113957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48374e6d-e275-4287-bc95-f4543cf77256 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:02.114404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48374e6d-e275-4287-bc95-f4543cf77256 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:02.114882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48374e6d-e275-4287-bc95-f4543cf77256 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:02.117917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48374e6d-e275-4287-bc95-f4543cf77256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:50:02.118402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48374e6d-e275-4287-bc95-f4543cf77256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:02.120487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48374e6d-e275-4287-bc95-f4543cf77256 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:02.162700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 825/1669] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:50:02.167401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9aca9-368b-49d6-ad69-e872a939f8f8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:02.181580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9aca9-368b-49d6-ad69-e872a939f8f8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:02.204209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2e535-15fe-402f-bebf-bbd28e80583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:02.205166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2e535-15fe-402f-bebf-bbd28e80583f admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:02.212278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2e535-15fe-402f-bebf-bbd28e80583f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:02.212761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2e535-15fe-402f-bebf-bbd28e80583f admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:02.214331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2e535-15fe-402f-bebf-bbd28e80583f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:02.217194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2e535-15fe-402f-bebf-bbd28e80583f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:02.302906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 845/1670] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:02 2022] POST /identity/v3/projects => generated 357 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:50:02.307784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48374e6d-e275-4287-bc95-f4543cf77256 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:02.331167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48374e6d-e275-4287-bc95-f4543cf77256 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:02.491980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e582e9-03b6-4237-b624-575257e3f5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:02.493209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e582e9-03b6-4237-b624-575257e3f5ba admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:02.493506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e582e9-03b6-4237-b624-575257e3f5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:02.493790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e582e9-03b6-4237-b624-575257e3f5ba admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:02.494500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e582e9-03b6-4237-b624-575257e3f5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:02.496507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e582e9-03b6-4237-b624-575257e3f5ba admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:02.524873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 826/1671] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:02 2022] POST /identity/v3/users => generated 279 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:02.528331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2e535-15fe-402f-bebf-bbd28e80583f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:02.558895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2e535-15fe-402f-bebf-bbd28e80583f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:02.567356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f65e6-6df2-47d4-ac31-5660dc7419ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:02.568621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f65e6-6df2-47d4-ac31-5660dc7419ba admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:02.568621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f65e6-6df2-47d4-ac31-5660dc7419ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:02.569137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f65e6-6df2-47d4-ac31-5660dc7419ba admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:02.570026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f65e6-6df2-47d4-ac31-5660dc7419ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:02.572739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f65e6-6df2-47d4-ac31-5660dc7419ba admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:02.591998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 846/1672] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:02.596502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e582e9-03b6-4237-b624-575257e3f5ba admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:02.613475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e582e9-03b6-4237-b624-575257e3f5ba admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:02.618475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2d40b-4adb-4759-ab60-22daf62cde34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:02.619202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2d40b-4adb-4759-ab60-22daf62cde34 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:02.619491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2d40b-4adb-4759-ab60-22daf62cde34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:02.619767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2d40b-4adb-4759-ab60-22daf62cde34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:02.620502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2d40b-4adb-4759-ab60-22daf62cde34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:02.622163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2d40b-4adb-4759-ab60-22daf62cde34 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:02.650716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 827/1673] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:02.651608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f65e6-6df2-47d4-ac31-5660dc7419ba admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:02.665076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f65e6-6df2-47d4-ac31-5660dc7419ba admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:02.671781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbdd31-d5a6-458b-b344-ead68229334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:02.672968 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbdd31-d5a6-458b-b344-ead68229334f admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:02.673376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbdd31-d5a6-458b-b344-ead68229334f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:02.673763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbdd31-d5a6-458b-b344-ead68229334f admin admin] PATH_INFO: `/v3/projects/ed1fd50156824e9ba5f077dea1e56ab2/users/f74c6c2cad8a42de98bec9347adff173/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:02.694406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbdd31-d5a6-458b-b344-ead68229334f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1fd50156824e9ba5f077dea1e56ab2, user_id=f74c6c2cad8a42de98bec9347adff173, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:02.697910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbdd31-d5a6-458b-b344-ead68229334f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:02.727787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 847/1674] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:02 2022] PUT /identity/v3/projects/ed1fd50156824e9ba5f077dea1e56ab2/users/f74c6c2cad8a42de98bec9347adff173/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:02.729590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd65dcf-b161-44e9-a9ba-2df88916033f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:02.730075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd65dcf-b161-44e9-a9ba-2df88916033f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:02.730178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd65dcf-b161-44e9-a9ba-2df88916033f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:02.761598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-2dd65dcf-b161-44e9-a9ba-2df88916033f None None] MFA Rules not processed for user `f74c6c2cad8a42de98bec9347adff173`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:02.763019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2dd65dcf-b161-44e9-a9ba-2df88916033f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:02.834330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:02.841861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 828/1675] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:02 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:02.843170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbdd31-d5a6-458b-b344-ead68229334f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:02.923399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:02.930142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbdd31-d5a6-458b-b344-ead68229334f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:02.940537 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2ac02-43f2-4382-9443-83b9168b9ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:02.942273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2ac02-43f2-4382-9443-83b9168b9ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:02.942789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2ac02-43f2-4382-9443-83b9168b9ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:02.943304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2ac02-43f2-4382-9443-83b9168b9ad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:02.944820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2ac02-43f2-4382-9443-83b9168b9ad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:02.947302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2ac02-43f2-4382-9443-83b9168b9ad5 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:02.992437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 848/1676] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:02 2022] POST /identity/v3/projects => generated 357 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:02.996491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd65dcf-b161-44e9-a9ba-2df88916033f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:03.015403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd65dcf-b161-44e9-a9ba-2df88916033f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:03.065740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36266a04-4646-4c5a-ac4b-6f15bb1aec3a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:03.065985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36266a04-4646-4c5a-ac4b-6f15bb1aec3a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:03.066205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36266a04-4646-4c5a-ac4b-6f15bb1aec3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:03.133492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-36266a04-4646-4c5a-ac4b-6f15bb1aec3a None None] MFA Rules not processed for user `3af6df3efbb74db4958654503e17d82e`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:03.137638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-36266a04-4646-4c5a-ac4b-6f15bb1aec3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:03.184989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44f454-ddbf-42c9-8662-6a1e7bfdc2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:03.186088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44f454-ddbf-42c9-8662-6a1e7bfdc2c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:03.186381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44f454-ddbf-42c9-8662-6a1e7bfdc2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:03.187128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44f454-ddbf-42c9-8662-6a1e7bfdc2c2 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:03.188081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44f454-ddbf-42c9-8662-6a1e7bfdc2c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:03.190680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44f454-ddbf-42c9-8662-6a1e7bfdc2c2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:03.247909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 829/1677] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:02 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:03.252798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44f454-ddbf-42c9-8662-6a1e7bfdc2c2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:03.271652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44f454-ddbf-42c9-8662-6a1e7bfdc2c2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:03.283495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5133d1-b57f-4e1c-8f63-780c22513cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:03.284189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5133d1-b57f-4e1c-8f63-780c22513cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:03.284477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5133d1-b57f-4e1c-8f63-780c22513cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:03.284758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5133d1-b57f-4e1c-8f63-780c22513cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:03.285506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5133d1-b57f-4e1c-8f63-780c22513cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:03.293794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5133d1-b57f-4e1c-8f63-780c22513cd7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:03.326373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 849/1678] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:03.327515 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 830/1679] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:03.334211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36266a04-4646-4c5a-ac4b-6f15bb1aec3a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:03.337653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5133d1-b57f-4e1c-8f63-780c22513cd7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:03.342687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3b5133d1-b57f-4e1c-8f63-780c22513cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:03.373910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36266a04-4646-4c5a-ac4b-6f15bb1aec3a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:03.392292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc445d-ecb9-4360-a3ee-272b4d0e9660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:03.393388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc445d-ecb9-4360-a3ee-272b4d0e9660 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:03.394006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc445d-ecb9-4360-a3ee-272b4d0e9660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:03.400599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc445d-ecb9-4360-a3ee-272b4d0e9660 admin admin] PATH_INFO: `/v3/projects/68fb14a59e0c4949ad7259fb7eb4b3ac/users/78e91027e2404b5b9d2b14b509d35a77/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:03.449816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc445d-ecb9-4360-a3ee-272b4d0e9660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68fb14a59e0c4949ad7259fb7eb4b3ac, user_id=78e91027e2404b5b9d2b14b509d35a77, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:03.452913 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc445d-ecb9-4360-a3ee-272b4d0e9660 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:03.491625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 850/1680] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:03 2022] PUT /identity/v3/projects/68fb14a59e0c4949ad7259fb7eb4b3ac/users/78e91027e2404b5b9d2b14b509d35a77/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:03.495515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc445d-ecb9-4360-a3ee-272b4d0e9660 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:03.554922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5133d1-b57f-4e1c-8f63-780c22513cd7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:03.597503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc445d-ecb9-4360-a3ee-272b4d0e9660 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:03.604116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163eacc8-9f28-45fe-a75b-81e3a5d60e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:03.605576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163eacc8-9f28-45fe-a75b-81e3a5d60e82 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:03.606059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163eacc8-9f28-45fe-a75b-81e3a5d60e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:03.606502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163eacc8-9f28-45fe-a75b-81e3a5d60e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:03.607716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163eacc8-9f28-45fe-a75b-81e3a5d60e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:03.610295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163eacc8-9f28-45fe-a75b-81e3a5d60e82 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:03.633861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 851/1681] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:03.636775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163eacc8-9f28-45fe-a75b-81e3a5d60e82 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:03.658147 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163eacc8-9f28-45fe-a75b-81e3a5d60e82 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:03.674032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77426917-41d4-48b8-b5eb-a34b07890114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:03.676100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77426917-41d4-48b8-b5eb-a34b07890114 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:03.676550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77426917-41d4-48b8-b5eb-a34b07890114 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:03.676977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77426917-41d4-48b8-b5eb-a34b07890114 admin admin] PATH_INFO: `/v3/projects/68fb14a59e0c4949ad7259fb7eb4b3ac/users/78e91027e2404b5b9d2b14b509d35a77/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:03.680455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77426917-41d4-48b8-b5eb-a34b07890114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68fb14a59e0c4949ad7259fb7eb4b3ac, user_id=78e91027e2404b5b9d2b14b509d35a77, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:03.684008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77426917-41d4-48b8-b5eb-a34b07890114 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:03.702087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 852/1682] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:03 2022] PUT /identity/v3/projects/68fb14a59e0c4949ad7259fb7eb4b3ac/users/78e91027e2404b5b9d2b14b509d35a77/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:03.713057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c8c53-b9ee-4116-944f-4ffda2d4333e None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:03.713343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c8c53-b9ee-4116-944f-4ffda2d4333e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:03.713988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c8c53-b9ee-4116-944f-4ffda2d4333e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:03.722859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd17269-6413-4f5b-8b40-081568ba65b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:03.723628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd17269-6413-4f5b-8b40-081568ba65b4 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:03.723912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd17269-6413-4f5b-8b40-081568ba65b4 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:03.724274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd17269-6413-4f5b-8b40-081568ba65b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:03.751421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:03.770032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-738c8c53-b9ee-4116-944f-4ffda2d4333e None None] MFA Rules not processed for user `78e91027e2404b5b9d2b14b509d35a77`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:03.772832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-738c8c53-b9ee-4116-944f-4ffda2d4333e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:03.819753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:03.837936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd17269-6413-4f5b-8b40-081568ba65b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:03.839756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd17269-6413-4f5b-8b40-081568ba65b4 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:03.860004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 831/1683] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:03.978378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 853/1684] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:03 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:03.990229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd17269-6413-4f5b-8b40-081568ba65b4 service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:03.992749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fdd17269-6413-4f5b-8b40-081568ba65b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:04.104495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236dd7d-d4a5-4c0e-85cb-8eff37dd4af1 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:04.104877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236dd7d-d4a5-4c0e-85cb-8eff37dd4af1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:04.105249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236dd7d-d4a5-4c0e-85cb-8eff37dd4af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:04.133606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-1236dd7d-d4a5-4c0e-85cb-8eff37dd4af1 None None] MFA Rules not processed for user `434c81b82dee404f8d143f1bdf56d3ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:04.134953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1236dd7d-d4a5-4c0e-85cb-8eff37dd4af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:04.135788 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd17269-6413-4f5b-8b40-081568ba65b4 service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:04.327103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 854/1685] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:04.331994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3093be-6acb-4ffc-8ac6-b93bab63c5cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:04.332777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3093be-6acb-4ffc-8ac6-b93bab63c5cb service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:04.333099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3093be-6acb-4ffc-8ac6-b93bab63c5cb service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:04.333443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3093be-6acb-4ffc-8ac6-b93bab63c5cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:04.354770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3093be-6acb-4ffc-8ac6-b93bab63c5cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:04.357531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3093be-6acb-4ffc-8ac6-b93bab63c5cb service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:04.381668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 832/1686] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:03 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:04.517291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236dd7d-d4a5-4c0e-85cb-8eff37dd4af1 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:04.536798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236dd7d-d4a5-4c0e-85cb-8eff37dd4af1 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:04.543681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b1b68-833d-4d70-8d36-6d47245c2e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:04.544275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b1b68-833d-4d70-8d36-6d47245c2e03 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:04.544563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b1b68-833d-4d70-8d36-6d47245c2e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:04.545327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b1b68-833d-4d70-8d36-6d47245c2e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:04.638646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b1b68-833d-4d70-8d36-6d47245c2e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:04.640215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b1b68-833d-4d70-8d36-6d47245c2e03 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:04.660494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 855/1687] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:04 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:04.850880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef28f-3dce-4e25-aad9-a8f1fae0ee1a None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:04.851837 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef28f-3dce-4e25-aad9-a8f1fae0ee1a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:04.852309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef28f-3dce-4e25-aad9-a8f1fae0ee1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:04.945813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-64eef28f-3dce-4e25-aad9-a8f1fae0ee1a None None] MFA Rules not processed for user `fb838f931b7748e2baf6c2315d3cbd75`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:04.948344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-64eef28f-3dce-4e25-aad9-a8f1fae0ee1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:05.105455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce4c3f-1f25-4db7-99a1-c50b94f16b2b None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:05.105754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce4c3f-1f25-4db7-99a1-c50b94f16b2b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:05.106059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce4c3f-1f25-4db7-99a1-c50b94f16b2b None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:05.108122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 856/1688] 10.222.0.62 () {54 vars in 912 bytes} [Wed Jun 15 15:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:50:05.121440 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11586f-5f7d-47fa-9930-c2b2d2744353 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:05.121732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11586f-5f7d-47fa-9930-c2b2d2744353 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:05.122017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11586f-5f7d-47fa-9930-c2b2d2744353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:05.126203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 833/1689] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:05.145269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef28f-3dce-4e25-aad9-a8f1fae0ee1a None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:05.152023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:05.154643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-af11586f-5f7d-47fa-9930-c2b2d2744353 None None] MFA Rules not processed for user `bf1915b9919442d692bcedf25f53320c`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:05.159634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-af11586f-5f7d-47fa-9930-c2b2d2744353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:05.254593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef28f-3dce-4e25-aad9-a8f1fae0ee1a None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:05.264673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 857/1690] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:05.268786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9c02d-4bb2-45ce-9086-1d4b83bc5205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:05.278413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9c02d-4bb2-45ce-9086-1d4b83bc5205 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:05.278704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9c02d-4bb2-45ce-9086-1d4b83bc5205 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:05.279034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9c02d-4bb2-45ce-9086-1d4b83bc5205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:05.279830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11586f-5f7d-47fa-9930-c2b2d2744353 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:05.281643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-af11586f-5f7d-47fa-9930-c2b2d2744353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:05.298434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11586f-5f7d-47fa-9930-c2b2d2744353 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:05.303199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9c02d-4bb2-45ce-9086-1d4b83bc5205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:05.304958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9c02d-4bb2-45ce-9086-1d4b83bc5205 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:05.306717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dbd92-14b6-4c3d-b0fd-81a48813f784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:05.308004 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dbd92-14b6-4c3d-b0fd-81a48813f784 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:05.308362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dbd92-14b6-4c3d-b0fd-81a48813f784 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:05.308799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dbd92-14b6-4c3d-b0fd-81a48813f784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:05.316108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:05.322422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35dbd92-14b6-4c3d-b0fd-81a48813f784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:05.324391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35dbd92-14b6-4c3d-b0fd-81a48813f784 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:05.338889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 858/1691] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:05.346970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 834/1692] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:50:07.548595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f0b0c-71a4-4b7b-8b07-45be50c3a4de None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:07.551517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f0b0c-71a4-4b7b-8b07-45be50c3a4de None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:07.554437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f0b0c-71a4-4b7b-8b07-45be50c3a4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:07.607747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-db0f0b0c-71a4-4b7b-8b07-45be50c3a4de None None] MFA Rules not processed for user `3af6df3efbb74db4958654503e17d82e`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:07.616459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-db0f0b0c-71a4-4b7b-8b07-45be50c3a4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:07.873664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 859/1693] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:08.353052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc25a51-38e3-4d03-8467-d8f942a960bc None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:08.354769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc25a51-38e3-4d03-8467-d8f942a960bc None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:08.354769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc25a51-38e3-4d03-8467-d8f942a960bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:08.395148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8cc25a51-38e3-4d03-8467-d8f942a960bc None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:08.399950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8cc25a51-38e3-4d03-8467-d8f942a960bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:08.513154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 835/1694] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:50:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:08.869156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f0b0c-71a4-4b7b-8b07-45be50c3a4de None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:08.870758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-db0f0b0c-71a4-4b7b-8b07-45be50c3a4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:09.051128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f0b0c-71a4-4b7b-8b07-45be50c3a4de None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:09.066972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b85ab-9bbc-41c2-bb4d-009424ceff12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:09.069354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b85ab-9bbc-41c2-bb4d-009424ceff12 service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:09.069663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b85ab-9bbc-41c2-bb4d-009424ceff12 service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:09.069947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b85ab-9bbc-41c2-bb4d-009424ceff12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:09.091526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b85ab-9bbc-41c2-bb4d-009424ceff12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:09.098202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b85ab-9bbc-41c2-bb4d-009424ceff12 service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:09.131565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 860/1695] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:50:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:50:11.192937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc25a51-38e3-4d03-8467-d8f942a960bc None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:11.209776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc25a51-38e3-4d03-8467-d8f942a960bc None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:11.237082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:11.238700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:11.239158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:11.239311 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:11.247656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:11.480010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:11.482341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:11.498891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 836/1696] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:11.709248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7a651-5f0a-431e-a04b-6d8c2d218b91 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:11.709780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7a651-5f0a-431e-a04b-6d8c2d218b91 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:11.709924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7a651-5f0a-431e-a04b-6d8c2d218b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:11.798237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-56a7a651-5f0a-431e-a04b-6d8c2d218b91 None None] MFA Rules not processed for user `bf1915b9919442d692bcedf25f53320c`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:11.814760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-56a7a651-5f0a-431e-a04b-6d8c2d218b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:11.852657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 861/1697] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Jun 15 15:50:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:11.897218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:11.899466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:11.927641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19853d40-7878-482a-b548-1db93e7a5840 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:11.946961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a3b9c-3fc4-4ea5-8147-804d643b6849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:11.948005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a3b9c-3fc4-4ea5-8147-804d643b6849 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:11.948005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a3b9c-3fc4-4ea5-8147-804d643b6849 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:11.948150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a3b9c-3fc4-4ea5-8147-804d643b6849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:11.969207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a3b9c-3fc4-4ea5-8147-804d643b6849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:11.975651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a3b9c-3fc4-4ea5-8147-804d643b6849 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:12.018350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 837/1698] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:12.091618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7a651-5f0a-431e-a04b-6d8c2d218b91 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:12.093393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-56a7a651-5f0a-431e-a04b-6d8c2d218b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:12.144592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327b079-2abb-40d5-9245-252eb0f82a63 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:12.144948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327b079-2abb-40d5-9245-252eb0f82a63 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:12.145354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327b079-2abb-40d5-9245-252eb0f82a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:12.186014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c327b079-2abb-40d5-9245-252eb0f82a63 None None] MFA Rules not processed for user `fb838f931b7748e2baf6c2315d3cbd75`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:12.188296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c327b079-2abb-40d5-9245-252eb0f82a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:12.239889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 838/1699] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:12.336066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7a651-5f0a-431e-a04b-6d8c2d218b91 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:12.369083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:12.381332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:12.382820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:12.383395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] PATH_INFO: `/v3/users/a37287e68cdd4bc4b8ec9d20a1daec8c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:12.386495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37287e68cdd4bc4b8ec9d20a1daec8c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:12.394961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:12.508809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:12.606082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37287e68cdd4bc4b8ec9d20a1daec8c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:12.641789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37287e68cdd4bc4b8ec9d20a1daec8c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:12.661340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37287e68cdd4bc4b8ec9d20a1daec8c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:12.663036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37287e68cdd4bc4b8ec9d20a1daec8c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:12.709251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37287e68cdd4bc4b8ec9d20a1daec8c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:12.729833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 862/1700] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:12 2022] DELETE /identity/v3/users/a37287e68cdd4bc4b8ec9d20a1daec8c => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:12.731589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1235d09-f69a-40a6-99bb-d57d8a08793a None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:12.732366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1235d09-f69a-40a6-99bb-d57d8a08793a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:12.732366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1235d09-f69a-40a6-99bb-d57d8a08793a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:12.792762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d1235d09-f69a-40a6-99bb-d57d8a08793a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:12.795808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d1235d09-f69a-40a6-99bb-d57d8a08793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:12.820143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:13.009048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 839/1701] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:13.025381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:13.026873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:13.189044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a2b46-ed12-4144-9c22-3313f9fe014e admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:13.193479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f5b2b-6838-419e-94cc-1d73560081ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:13.194192 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f5b2b-6838-419e-94cc-1d73560081ec service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:13.194478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f5b2b-6838-419e-94cc-1d73560081ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:13.194828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f5b2b-6838-419e-94cc-1d73560081ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:13.207808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f5b2b-6838-419e-94cc-1d73560081ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:13.215782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f5b2b-6838-419e-94cc-1d73560081ec service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:13.229728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 863/1702] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:15.627288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1235d09-f69a-40a6-99bb-d57d8a08793a None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:15.630055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d1235d09-f69a-40a6-99bb-d57d8a08793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:15.657249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1235d09-f69a-40a6-99bb-d57d8a08793a None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:15.691229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:15.699395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:15.702453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:15.703640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] PATH_INFO: `/v3/projects/f6300459e2d1437eb386269c9e0c8a8f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:15.709988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6300459e2d1437eb386269c9e0c8a8f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:15.716815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:15.879726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09485eb9-a2a9-4c84-a161-84ed751213e0 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:15.880523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09485eb9-a2a9-4c84-a161-84ed751213e0 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:15.880958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09485eb9-a2a9-4c84-a161-84ed751213e0 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:15.882475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6300459e2d1437eb386269c9e0c8a8f', 'request_id': 'req-0667702b-c147-4762-aa91-2de16f4fdc60'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:15.883121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 864/1703] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:50:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:50:15.884337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09485eb9-a2a9-4c84-a161-84ed751213e0 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:15.886496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-09485eb9-a2a9-4c84-a161-84ed751213e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:15.909826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6300459e2d1437eb386269c9e0c8a8f', 'request_id': 'req-0667702b-c147-4762-aa91-2de16f4fdc60'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:15.933596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6300459e2d1437eb386269c9e0c8a8f', 'request_id': 'req-0667702b-c147-4762-aa91-2de16f4fdc60'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:15.960703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] The token cache is being invalidate because project f6300459e2d1437eb386269c9e0c8a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:15.961200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:15.964497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0667702b-c147-4762-aa91-2de16f4fdc60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:15.967081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 840/1704] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:15 2022] DELETE /identity/v3/projects/f6300459e2d1437eb386269c9e0c8a8f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:15.971599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f0296-741b-40d2-ac58-3f0d4b4f88e4 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:15.974379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f0296-741b-40d2-ac58-3f0d4b4f88e4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:15.975155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f0296-741b-40d2-ac58-3f0d4b4f88e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:16.031302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8c7f0296-741b-40d2-ac58-3f0d4b4f88e4 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:16.035132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8c7f0296-741b-40d2-ac58-3f0d4b4f88e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:16.067195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09485eb9-a2a9-4c84-a161-84ed751213e0 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:16.094971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:16.109937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:16.113016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:16.113505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] PATH_INFO: `/v3/users/fe4516f840a2496d84091b426f056fe8` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:16.122971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe4516f840a2496d84091b426f056fe8)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:16.125319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:16.173922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 841/1705] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:50:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:16.176186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bc12-a2ef-425d-b5cb-cc8edb2be00e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:16.178802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bc12-a2ef-425d-b5cb-cc8edb2be00e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:16.179354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bc12-a2ef-425d-b5cb-cc8edb2be00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:16.208008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f8b5bc12-a2ef-425d-b5cb-cc8edb2be00e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:16.211417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f8b5bc12-a2ef-425d-b5cb-cc8edb2be00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:16.232436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 842/1706] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:16.247833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8b711-3ffd-4a91-b9dc-fe5aac52b0f4 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:16.248119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8b711-3ffd-4a91-b9dc-fe5aac52b0f4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:16.248403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8b711-3ffd-4a91-b9dc-fe5aac52b0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:16.315453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:16.319246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe4516f840a2496d84091b426f056fe8'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:16.321185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-a4d8b711-3ffd-4a91-b9dc-fe5aac52b0f4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:16.322527 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a4d8b711-3ffd-4a91-b9dc-fe5aac52b0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:16.348529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe4516f840a2496d84091b426f056fe8'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:16.363515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe4516f840a2496d84091b426f056fe8'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:16.364700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe4516f840a2496d84091b426f056fe8'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:16.381799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 843/1707] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:16.381799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8b711-3ffd-4a91-b9dc-fe5aac52b0f4 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:16.386972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a4d8b711-3ffd-4a91-b9dc-fe5aac52b0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:16.388200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe4516f840a2496d84091b426f056fe8'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:16.407284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 865/1708] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:15 2022] DELETE /identity/v3/users/fe4516f840a2496d84091b426f056fe8 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:16.407284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:16.408832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:16.451635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8b711-3ffd-4a91-b9dc-fe5aac52b0f4 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:16.481679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:16.679964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4ad18-3d0a-4fe6-819c-65b1a8612791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:16.680640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4ad18-3d0a-4fe6-819c-65b1a8612791 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:16.680640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4ad18-3d0a-4fe6-819c-65b1a8612791 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:16.680828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4ad18-3d0a-4fe6-819c-65b1a8612791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:16.684562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa4ee8-71ad-49b5-b46e-2df327f55b61 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:16.697652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-bec4ad18-3d0a-4fe6-819c-65b1a8612791 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:16.716336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183c11b-684e-4ada-87cc-e2dea1d1cd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:16.717066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183c11b-684e-4ada-87cc-e2dea1d1cd27 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:16.717325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183c11b-684e-4ada-87cc-e2dea1d1cd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:16.717605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183c11b-684e-4ada-87cc-e2dea1d1cd27 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:16.718390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183c11b-684e-4ada-87cc-e2dea1d1cd27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:50:16.718715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183c11b-684e-4ada-87cc-e2dea1d1cd27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:16.737466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183c11b-684e-4ada-87cc-e2dea1d1cd27 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:16.771656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4ad18-3d0a-4fe6-819c-65b1a8612791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:16.773274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4ad18-3d0a-4fe6-819c-65b1a8612791 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:16.824940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 866/1709] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:50:16.829352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63644b-8cfd-4049-a970-9926644e41da None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:16.829642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63644b-8cfd-4049-a970-9926644e41da None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:16.829926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63644b-8cfd-4049-a970-9926644e41da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:16.857561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 844/1710] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:16.858506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4ad18-3d0a-4fe6-819c-65b1a8612791 service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:16.896108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4ad18-3d0a-4fe6-819c-65b1a8612791 service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:16.897901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-fc63644b-8cfd-4049-a970-9926644e41da None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:16.899173 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fc63644b-8cfd-4049-a970-9926644e41da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:16.912741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc985c-4814-4b18-bdc1-c9fd2ea9ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:16.913428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc985c-4814-4b18-bdc1-c9fd2ea9ec3a admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:16.913714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc985c-4814-4b18-bdc1-c9fd2ea9ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:16.913993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc985c-4814-4b18-bdc1-c9fd2ea9ec3a admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:16.914704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc985c-4814-4b18-bdc1-c9fd2ea9ec3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:16.918691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc985c-4814-4b18-bdc1-c9fd2ea9ec3a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:16.936313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 867/1711] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:16.983870 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63644b-8cfd-4049-a970-9926644e41da None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:16.986489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fc63644b-8cfd-4049-a970-9926644e41da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:17.042369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 845/1712] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:16 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:17.052506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc985c-4814-4b18-bdc1-c9fd2ea9ec3a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:17.196525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63644b-8cfd-4049-a970-9926644e41da None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:17.307500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc985c-4814-4b18-bdc1-c9fd2ea9ec3a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:17.314808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b9121-15ff-47cf-99cf-c1182efe207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:17.315620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b9121-15ff-47cf-99cf-c1182efe207a admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:17.315903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b9121-15ff-47cf-99cf-c1182efe207a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:17.316181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b9121-15ff-47cf-99cf-c1182efe207a admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:17.316875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b9121-15ff-47cf-99cf-c1182efe207a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:17.319137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b9121-15ff-47cf-99cf-c1182efe207a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:17.331497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475ffb7-61c2-42dd-a073-3b750e565fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:17.332188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475ffb7-61c2-42dd-a073-3b750e565fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:17.332471 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475ffb7-61c2-42dd-a073-3b750e565fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:17.332757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475ffb7-61c2-42dd-a073-3b750e565fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:17.356577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475ffb7-61c2-42dd-a073-3b750e565fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:17.357892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475ffb7-61c2-42dd-a073-3b750e565fa4 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:17.358572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 846/1713] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:17 2022] POST /identity/v3/users => generated 283 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:17.360877 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b9121-15ff-47cf-99cf-c1182efe207a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:17.380548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 868/1714] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:50:17.417924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b9121-15ff-47cf-99cf-c1182efe207a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:17.427044 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd42079-1deb-4b9c-a97e-5af2702e89f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:17.427913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd42079-1deb-4b9c-a97e-5af2702e89f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:17.428200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd42079-1deb-4b9c-a97e-5af2702e89f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:17.428537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd42079-1deb-4b9c-a97e-5af2702e89f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:17.429331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd42079-1deb-4b9c-a97e-5af2702e89f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:17.433690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd42079-1deb-4b9c-a97e-5af2702e89f6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:17.475404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 847/1715] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:17.480271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475ffb7-61c2-42dd-a073-3b750e565fa4 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:17.497321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475ffb7-61c2-42dd-a073-3b750e565fa4 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:17.514609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331280c-c07e-4236-80dd-4a5f4d947d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:17.516930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331280c-c07e-4236-80dd-4a5f4d947d8a admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:17.517338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331280c-c07e-4236-80dd-4a5f4d947d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:17.517849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331280c-c07e-4236-80dd-4a5f4d947d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:17.518741 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331280c-c07e-4236-80dd-4a5f4d947d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:17.524475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331280c-c07e-4236-80dd-4a5f4d947d8a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:17.530210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:17.553066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 869/1716] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:17.559008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd42079-1deb-4b9c-a97e-5af2702e89f6 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:17.574285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd42079-1deb-4b9c-a97e-5af2702e89f6 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:17.588129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2b015-6bf6-4f6f-86b1-c6c768917dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:17.589298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2b015-6bf6-4f6f-86b1-c6c768917dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:17.590352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2b015-6bf6-4f6f-86b1-c6c768917dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:17.591028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2b015-6bf6-4f6f-86b1-c6c768917dd9 admin admin] PATH_INFO: `/v3/projects/eaf6cc06569043e0a21ac310480902f6/users/736308d2b1954534a22e80248c514635/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:17.650541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2b015-6bf6-4f6f-86b1-c6c768917dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf6cc06569043e0a21ac310480902f6, user_id=736308d2b1954534a22e80248c514635, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:17.653401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2b015-6bf6-4f6f-86b1-c6c768917dd9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:17.689014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 848/1717] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:17 2022] PUT /identity/v3/projects/eaf6cc06569043e0a21ac310480902f6/users/736308d2b1954534a22e80248c514635/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:17.699168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48fb38-f0e8-434c-b67e-67091e586f0a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:17.699922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48fb38-f0e8-434c-b67e-67091e586f0a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:17.700759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48fb38-f0e8-434c-b67e-67091e586f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:17.762781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2f48fb38-f0e8-434c-b67e-67091e586f0a None None] MFA Rules not processed for user `736308d2b1954534a22e80248c514635`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:17.768453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2f48fb38-f0e8-434c-b67e-67091e586f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:18.018169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 870/1718] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:18.022991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2b015-6bf6-4f6f-86b1-c6c768917dd9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:18.155655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:18.249698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2b015-6bf6-4f6f-86b1-c6c768917dd9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:18.264079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60268907-6f9c-43c8-a554-50017dc7ea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:18.265006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60268907-6f9c-43c8-a554-50017dc7ea99 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:18.268647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60268907-6f9c-43c8-a554-50017dc7ea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:18.270008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60268907-6f9c-43c8-a554-50017dc7ea99 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:18.270967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60268907-6f9c-43c8-a554-50017dc7ea99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:18.276004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60268907-6f9c-43c8-a554-50017dc7ea99 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:18.331896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48fb38-f0e8-434c-b67e-67091e586f0a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:18.344193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 849/1719] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:18 2022] POST /identity/v3/projects => generated 365 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:18.359061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48fb38-f0e8-434c-b67e-67091e586f0a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:18.732422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ada4be-3006-4d2f-b008-ba68f35b26a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:18.735169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ada4be-3006-4d2f-b008-ba68f35b26a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:18.738398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ada4be-3006-4d2f-b008-ba68f35b26a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:18.740760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ada4be-3006-4d2f-b008-ba68f35b26a8 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:18.742387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ada4be-3006-4d2f-b008-ba68f35b26a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:18.758745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ada4be-3006-4d2f-b008-ba68f35b26a8 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:18.878792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 871/1720] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:18 2022] POST /identity/v3/users => generated 289 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:18.882021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60268907-6f9c-43c8-a554-50017dc7ea99 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:18.903107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60268907-6f9c-43c8-a554-50017dc7ea99 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:18.924143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf994df-eafb-4da7-a80b-b507d8616789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:18.924902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf994df-eafb-4da7-a80b-b507d8616789 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:18.925177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf994df-eafb-4da7-a80b-b507d8616789 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:18.927480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf994df-eafb-4da7-a80b-b507d8616789 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:18.930224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf994df-eafb-4da7-a80b-b507d8616789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:18.943240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf994df-eafb-4da7-a80b-b507d8616789 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:19.002964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ada4be-3006-4d2f-b008-ba68f35b26a8 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:19.008416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 850/1721] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:19.021610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ada4be-3006-4d2f-b008-ba68f35b26a8 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:19.048729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7ce5c-b57a-47e6-a285-932d58e6e1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:19.049829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7ce5c-b57a-47e6-a285-932d58e6e1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:19.050306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7ce5c-b57a-47e6-a285-932d58e6e1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:19.050751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7ce5c-b57a-47e6-a285-932d58e6e1a4 admin admin] PATH_INFO: `/v3/projects/8b15a8b03b814fea86d6ea331ebaa111/users/65fcb622d7e54a3e89a5a032d3f17288/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:19.119739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7ce5c-b57a-47e6-a285-932d58e6e1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b15a8b03b814fea86d6ea331ebaa111, user_id=65fcb622d7e54a3e89a5a032d3f17288, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:19.122468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7ce5c-b57a-47e6-a285-932d58e6e1a4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:19.175135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 872/1722] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:19 2022] PUT /identity/v3/projects/8b15a8b03b814fea86d6ea331ebaa111/users/65fcb622d7e54a3e89a5a032d3f17288/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:19.178788 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf994df-eafb-4da7-a80b-b507d8616789 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:19.197837 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf994df-eafb-4da7-a80b-b507d8616789 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:19.489415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60bcd6-19af-433a-8d4f-802d3467995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:19.494776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60bcd6-19af-433a-8d4f-802d3467995c admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:19.495094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60bcd6-19af-433a-8d4f-802d3467995c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:19.496595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60bcd6-19af-433a-8d4f-802d3467995c admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:19.500507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60bcd6-19af-433a-8d4f-802d3467995c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:19.541633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60bcd6-19af-433a-8d4f-802d3467995c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:19.591855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7ce5c-b57a-47e6-a285-932d58e6e1a4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:19.613149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 851/1723] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:19.615570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7ce5c-b57a-47e6-a285-932d58e6e1a4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:19.632242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8509c8-82e4-4f27-bbe0-c1b54aaa4ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:19.633564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8509c8-82e4-4f27-bbe0-c1b54aaa4ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:19.634120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8509c8-82e4-4f27-bbe0-c1b54aaa4ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:19.636583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8509c8-82e4-4f27-bbe0-c1b54aaa4ea4 admin admin] PATH_INFO: `/v3/projects/8b15a8b03b814fea86d6ea331ebaa111/users/65fcb622d7e54a3e89a5a032d3f17288/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:19.640882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8509c8-82e4-4f27-bbe0-c1b54aaa4ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b15a8b03b814fea86d6ea331ebaa111, user_id=65fcb622d7e54a3e89a5a032d3f17288, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:19.649230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8509c8-82e4-4f27-bbe0-c1b54aaa4ea4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:19.686605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 873/1724] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:19 2022] PUT /identity/v3/projects/8b15a8b03b814fea86d6ea331ebaa111/users/65fcb622d7e54a3e89a5a032d3f17288/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:19.688504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4135f2-64ca-46e3-9739-6b9507c51aec None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:19.689029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4135f2-64ca-46e3-9739-6b9507c51aec None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:19.689483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4135f2-64ca-46e3-9739-6b9507c51aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:19.771758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ad4135f2-64ca-46e3-9739-6b9507c51aec None None] MFA Rules not processed for user `65fcb622d7e54a3e89a5a032d3f17288`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:19.775453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ad4135f2-64ca-46e3-9739-6b9507c51aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:19.981614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 852/1725] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:19.997490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8509c8-82e4-4f27-bbe0-c1b54aaa4ea4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:20.000687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8e8509c8-82e4-4f27-bbe0-c1b54aaa4ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:20.113333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:20.142526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8509c8-82e4-4f27-bbe0-c1b54aaa4ea4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:20.148118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833b87b-2cfa-4534-8dd9-de2ef3a47935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:20.149135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833b87b-2cfa-4534-8dd9-de2ef3a47935 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:20.149630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833b87b-2cfa-4534-8dd9-de2ef3a47935 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:20.150546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833b87b-2cfa-4534-8dd9-de2ef3a47935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:20.162520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833b87b-2cfa-4534-8dd9-de2ef3a47935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:20.164328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833b87b-2cfa-4534-8dd9-de2ef3a47935 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:20.177557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 874/1726] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:21.246087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4135f2-64ca-46e3-9739-6b9507c51aec None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:21.334163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:21.477896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4135f2-64ca-46e3-9739-6b9507c51aec None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:21.495093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:21.497006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:21.498643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:21.499148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] PATH_INFO: `/v3/users/736308d2b1954534a22e80248c514635` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:21.502669 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=736308d2b1954534a22e80248c514635)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:21.527879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:21.708593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '736308d2b1954534a22e80248c514635'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:21.721290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '736308d2b1954534a22e80248c514635'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:21.729395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833b87b-2cfa-4534-8dd9-de2ef3a47935 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:21.733165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1833b87b-2cfa-4534-8dd9-de2ef3a47935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:21.755886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '736308d2b1954534a22e80248c514635'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:21.757522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '736308d2b1954534a22e80248c514635'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:21.761023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a1faa4f4-545e-4a9d-959d-391edd51c0a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '736308d2b1954534a22e80248c514635'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:21.767067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833b87b-2cfa-4534-8dd9-de2ef3a47935 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:21.784162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:21.786763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:21.792885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:21.792885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] PATH_INFO: `/v3/projects/c5cc94afeb35472a9f2d9cb52083da46` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:21.792885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5cc94afeb35472a9f2d9cb52083da46)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:21.801507 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:21.824403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 853/1727] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:21 2022] DELETE /identity/v3/users/736308d2b1954534a22e80248c514635 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:21.829312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffff25-cffe-4291-a3b7-5833c4f4355c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:21.829603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffff25-cffe-4291-a3b7-5833c4f4355c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:21.830659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffff25-cffe-4291-a3b7-5833c4f4355c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:21.881081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ccffff25-cffe-4291-a3b7-5833c4f4355c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:21.884219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ccffff25-cffe-4291-a3b7-5833c4f4355c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:21.975844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5cc94afeb35472a9f2d9cb52083da46', 'request_id': 'req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.029489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5cc94afeb35472a9f2d9cb52083da46', 'request_id': 'req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.051145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5cc94afeb35472a9f2d9cb52083da46', 'request_id': 'req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.100642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] The token cache is being invalidate because project c5cc94afeb35472a9f2d9cb52083da46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:22.100642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.101711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.107356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 875/1728] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:21 2022] DELETE /identity/v3/projects/c5cc94afeb35472a9f2d9cb52083da46 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:22.116852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:22.122369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:22.181300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 854/1729] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:22.268008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffff25-cffe-4291-a3b7-5833c4f4355c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:22.270353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ccffff25-cffe-4291-a3b7-5833c4f4355c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:22.360239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961ea08-ffeb-406a-ba9e-2d644dbc5d8d admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:22.377329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:22.382516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:22.382848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:22.383132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] PATH_INFO: `/v3/users/9af376ff010644e7b5b59bb9abbaef82` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:22.392983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af376ff010644e7b5b59bb9abbaef82)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:22.397475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:22.492143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:22.505447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffff25-cffe-4291-a3b7-5833c4f4355c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:22.547110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af376ff010644e7b5b59bb9abbaef82'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.564093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af376ff010644e7b5b59bb9abbaef82'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.576553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af376ff010644e7b5b59bb9abbaef82'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.578423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af376ff010644e7b5b59bb9abbaef82'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.585352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:22.592461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8bd6b-2979-459a-bbc9-41392610c151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:22.594230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8bd6b-2979-459a-bbc9-41392610c151 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:22.594782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8bd6b-2979-459a-bbc9-41392610c151 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:22.595395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8bd6b-2979-459a-bbc9-41392610c151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:22.600114 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0ebfd116-4f2b-4c7f-8bb5-0039c3ec5000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af376ff010644e7b5b59bb9abbaef82'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:22.609019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 876/1730] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:22 2022] DELETE /identity/v3/users/9af376ff010644e7b5b59bb9abbaef82 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:22.613710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8bd6b-2979-459a-bbc9-41392610c151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:22.615380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8bd6b-2979-459a-bbc9-41392610c151 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:22.643725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 855/1731] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:50:24.184515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2f75-ba10-4b09-93bf-5daf44699d6f None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:24.185452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2f75-ba10-4b09-93bf-5daf44699d6f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:24.185894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2f75-ba10-4b09-93bf-5daf44699d6f None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:24.187975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 877/1732] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:50:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:50:24.198857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11890cb0-b202-420d-a29c-e223d267a261 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:24.199431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11890cb0-b202-420d-a29c-e223d267a261 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:24.199525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11890cb0-b202-420d-a29c-e223d267a261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:24.237017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-11890cb0-b202-420d-a29c-e223d267a261 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:24.240719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-11890cb0-b202-420d-a29c-e223d267a261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:24.355917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b2f75-ba10-4b09-93bf-5daf44699d6f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:24.362455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9e4b2f75-ba10-4b09-93bf-5daf44699d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:24.469185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 856/1733] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:24.515776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11890cb0-b202-420d-a29c-e223d267a261 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:24.517503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-11890cb0-b202-420d-a29c-e223d267a261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:24.622337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b2f75-ba10-4b09-93bf-5daf44699d6f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:24.633647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:24.634533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:24.634867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:24.635145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] PATH_INFO: `/v3/projects/53685cedcf4c429f8405362f51716015` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:24.636376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53685cedcf4c429f8405362f51716015)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:24.638753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:24.704811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11890cb0-b202-420d-a29c-e223d267a261 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:24.719257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53685cedcf4c429f8405362f51716015', 'request_id': 'req-b672b9b9-719f-4fd0-97d1-e650fb3d9710'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.730019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53685cedcf4c429f8405362f51716015', 'request_id': 'req-b672b9b9-719f-4fd0-97d1-e650fb3d9710'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.748236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53685cedcf4c429f8405362f51716015', 'request_id': 'req-b672b9b9-719f-4fd0-97d1-e650fb3d9710'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.770801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] The token cache is being invalidate because project 53685cedcf4c429f8405362f51716015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:24.771313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.772130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.773951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 878/1734] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:24 2022] DELETE /identity/v3/projects/53685cedcf4c429f8405362f51716015 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:24.774840 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:24.777924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:24.809806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:24.833542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:24.835158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:24.836283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:24.836283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] PATH_INFO: `/v3/projects/eaf6cc06569043e0a21ac310480902f6` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:24.837934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf6cc06569043e0a21ac310480902f6)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:24.840894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:24.910512 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b9b9-719f-4fd0-97d1-e650fb3d9710 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:24.927761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:24.929593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf6cc06569043e0a21ac310480902f6', 'request_id': 'req-98bc2129-fada-4fdc-9efe-43929a609292'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.938209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf6cc06569043e0a21ac310480902f6', 'request_id': 'req-98bc2129-fada-4fdc-9efe-43929a609292'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.946358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf6cc06569043e0a21ac310480902f6', 'request_id': 'req-98bc2129-fada-4fdc-9efe-43929a609292'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.958225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] The token cache is being invalidate because project eaf6cc06569043e0a21ac310480902f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:24.958631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.961664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:24.963555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 857/1735] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:24 2022] DELETE /identity/v3/projects/eaf6cc06569043e0a21ac310480902f6 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:24.967103 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:24.968535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:24.991127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:24.991799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:24.992090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:24.992390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:24.994846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:25.009080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:25.010697 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:25.028268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 879/1736] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:25.029823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:25.032637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:25.058312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc2129-fada-4fdc-9efe-43929a609292 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:25.066635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:25.068821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:25.069314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:25.069599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] PATH_INFO: `/v3/users/eedfa25f9e1249a8b13db83c5bc9c0f8` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:25.071025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eedfa25f9e1249a8b13db83c5bc9c0f8)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:25.073296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:25.151492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79580fc9-9342-4ef0-95a5-c27388fdf9ec service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:25.153066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eedfa25f9e1249a8b13db83c5bc9c0f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.170861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:25.172515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:25.173103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:25.174571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] PATH_INFO: `/v3/users/65fcb622d7e54a3e89a5a032d3f17288` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:25.175207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eedfa25f9e1249a8b13db83c5bc9c0f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.176405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65fcb622d7e54a3e89a5a032d3f17288)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:25.178814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:25.183972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eedfa25f9e1249a8b13db83c5bc9c0f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.184518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eedfa25f9e1249a8b13db83c5bc9c0f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.185139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eedfa25f9e1249a8b13db83c5bc9c0f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.198087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:25.208474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 858/1737] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:24 2022] DELETE /identity/v3/users/eedfa25f9e1249a8b13db83c5bc9c0f8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:25.235777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65fcb622d7e54a3e89a5a032d3f17288'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.244643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65fcb622d7e54a3e89a5a032d3f17288'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.255124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65fcb622d7e54a3e89a5a032d3f17288'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.255856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65fcb622d7e54a3e89a5a032d3f17288'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.270463 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65fcb622d7e54a3e89a5a032d3f17288'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:25.278135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 880/1738] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:25 2022] DELETE /identity/v3/users/65fcb622d7e54a3e89a5a032d3f17288 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:25.783935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:25.785610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:25.856219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595d52b-9996-4b41-b2f6-5c6d481701e4 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:25.979714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:25.979714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:25.980058 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:25.980486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] PATH_INFO: `/v3/projects/02af1123fc984294901d0d741f64b37a` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:25.981912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02af1123fc984294901d0d741f64b37a)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:25.985071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.013316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.020932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.069216 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ca7ac-c69a-4fc1-8408-781a27cc3021 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.087627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02af1123fc984294901d0d741f64b37a', 'request_id': 'req-981cd767-9a30-4e36-b1af-75023ee870a6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.093684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:26.112025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02af1123fc984294901d0d741f64b37a', 'request_id': 'req-981cd767-9a30-4e36-b1af-75023ee870a6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.129218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02af1123fc984294901d0d741f64b37a', 'request_id': 'req-981cd767-9a30-4e36-b1af-75023ee870a6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.149691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] The token cache is being invalidate because project 02af1123fc984294901d0d741f64b37a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:26.149985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.152572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-981cd767-9a30-4e36-b1af-75023ee870a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.154499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 859/1739] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:25 2022] DELETE /identity/v3/projects/02af1123fc984294901d0d741f64b37a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:26.161005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4691ee-49d6-48b1-912b-1f1824f421be None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.161551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4691ee-49d6-48b1-912b-1f1824f421be None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.161826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4691ee-49d6-48b1-912b-1f1824f421be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.193198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.194078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.200945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.201309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] PATH_INFO: `/v3/projects/8b15a8b03b814fea86d6ea331ebaa111` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.203779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b15a8b03b814fea86d6ea331ebaa111)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.207499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.216542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-dd4691ee-49d6-48b1-912b-1f1824f421be None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:26.218371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-dd4691ee-49d6-48b1-912b-1f1824f421be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.228581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 860/1740] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:26.237575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07c631-b551-47e6-a530-febfd2bcfee6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.237865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07c631-b551-47e6-a530-febfd2bcfee6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.238143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07c631-b551-47e6-a530-febfd2bcfee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.261211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-5d07c631-b551-47e6-a530-febfd2bcfee6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:26.262674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5d07c631-b551-47e6-a530-febfd2bcfee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.285447 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b15a8b03b814fea86d6ea331ebaa111', 'request_id': 'req-486b5072-e186-415a-8d4b-2d905887528f'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.286000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:26.300816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b15a8b03b814fea86d6ea331ebaa111', 'request_id': 'req-486b5072-e186-415a-8d4b-2d905887528f'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.311299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b15a8b03b814fea86d6ea331ebaa111', 'request_id': 'req-486b5072-e186-415a-8d4b-2d905887528f'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.325279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] The token cache is being invalidate because project 8b15a8b03b814fea86d6ea331ebaa111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:26.325572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.326719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.329250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 881/1741] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:26 2022] DELETE /identity/v3/projects/8b15a8b03b814fea86d6ea331ebaa111 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:26.329829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.331172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.339843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 861/1742] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:26.341585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171a95d-1d66-4537-9f38-225835331798 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.342025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171a95d-1d66-4537-9f38-225835331798 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.342906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171a95d-1d66-4537-9f38-225835331798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.347287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b5072-e186-415a-8d4b-2d905887528f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.352345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.353297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.353659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.354015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] PATH_INFO: `/v3/users/f74c6c2cad8a42de98bec9347adff173` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.355331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f74c6c2cad8a42de98bec9347adff173)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.357420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.361481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9171a95d-1d66-4537-9f38-225835331798 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:26.362722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9171a95d-1d66-4537-9f38-225835331798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.378824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 862/1743] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:26.378824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171a95d-1d66-4537-9f38-225835331798 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.383447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171a95d-1d66-4537-9f38-225835331798 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.388007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e1ede-93d8-4a44-a701-2d3c9faf1715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.388666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e1ede-93d8-4a44-a701-2d3c9faf1715 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.388948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e1ede-93d8-4a44-a701-2d3c9faf1715 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.389239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e1ede-93d8-4a44-a701-2d3c9faf1715 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.389962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e1ede-93d8-4a44-a701-2d3c9faf1715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:50:26.390832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e1ede-93d8-4a44-a701-2d3c9faf1715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.392498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e1ede-93d8-4a44-a701-2d3c9faf1715 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.406794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 863/1744] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:50:26.411556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f15167-94db-4994-80fc-267dc612b7f4 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.411738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f15167-94db-4994-80fc-267dc612b7f4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.412016 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f15167-94db-4994-80fc-267dc612b7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.418814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f74c6c2cad8a42de98bec9347adff173'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.427007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f74c6c2cad8a42de98bec9347adff173'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.430288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-93f15167-94db-4994-80fc-267dc612b7f4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:26.432349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-93f15167-94db-4994-80fc-267dc612b7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.439224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f74c6c2cad8a42de98bec9347adff173'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.440080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f74c6c2cad8a42de98bec9347adff173'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.440305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 864/1745] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:26.441071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f15167-94db-4994-80fc-267dc612b7f4 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.451914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f15167-94db-4994-80fc-267dc612b7f4 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.457132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69743c11-3591-49c5-bf22-80281eaa9e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.457806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69743c11-3591-49c5-bf22-80281eaa9e00 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.458092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69743c11-3591-49c5-bf22-80281eaa9e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.458371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69743c11-3591-49c5-bf22-80281eaa9e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.459112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69743c11-3591-49c5-bf22-80281eaa9e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.461006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69743c11-3591-49c5-bf22-80281eaa9e00 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.468051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f74c6c2cad8a42de98bec9347adff173'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:26.475933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 882/1746] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:26 2022] DELETE /identity/v3/users/f74c6c2cad8a42de98bec9347adff173 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:26.477038 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.478796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.493795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 865/1747] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:26.495472 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b751b20-bd3d-4fdd-8226-8fa6e7e3d34a None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.495763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b751b20-bd3d-4fdd-8226-8fa6e7e3d34a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.496045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b751b20-bd3d-4fdd-8226-8fa6e7e3d34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.510886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:26.515826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9b751b20-bd3d-4fdd-8226-8fa6e7e3d34a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:26.517509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9b751b20-bd3d-4fdd-8226-8fa6e7e3d34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.560276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50210c79-fc51-481e-bfd9-a72772ed8ab1 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.581828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 866/1748] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:26.582658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b751b20-bd3d-4fdd-8226-8fa6e7e3d34a None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.584141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9b751b20-bd3d-4fdd-8226-8fa6e7e3d34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.597007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b751b20-bd3d-4fdd-8226-8fa6e7e3d34a None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.602511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413aba6-c596-4609-a811-773ed6af08b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.603217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413aba6-c596-4609-a811-773ed6af08b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.603501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413aba6-c596-4609-a811-773ed6af08b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.603777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413aba6-c596-4609-a811-773ed6af08b9 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.604464 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413aba6-c596-4609-a811-773ed6af08b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.606398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413aba6-c596-4609-a811-773ed6af08b9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.623078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dbd76-f88f-4607-bc5f-ff4b2c0a6194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.623740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dbd76-f88f-4607-bc5f-ff4b2c0a6194 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.624020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dbd76-f88f-4607-bc5f-ff4b2c0a6194 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.624292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dbd76-f88f-4607-bc5f-ff4b2c0a6194 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.625034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dbd76-f88f-4607-bc5f-ff4b2c0a6194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:50:26.625339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dbd76-f88f-4607-bc5f-ff4b2c0a6194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.626952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dbd76-f88f-4607-bc5f-ff4b2c0a6194 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.627843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 867/1749] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:26.628652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413aba6-c596-4609-a811-773ed6af08b9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.630200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0413aba6-c596-4609-a811-773ed6af08b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.641733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 883/1750] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:50:26.642770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dbd76-f88f-4607-bc5f-ff4b2c0a6194 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.652542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dbd76-f88f-4607-bc5f-ff4b2c0a6194 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.658207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f9b3c-026e-4712-a3e0-fd35565a3d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.658908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f9b3c-026e-4712-a3e0-fd35565a3d89 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.659195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f9b3c-026e-4712-a3e0-fd35565a3d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.659475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f9b3c-026e-4712-a3e0-fd35565a3d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.660225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f9b3c-026e-4712-a3e0-fd35565a3d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.662098 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f9b3c-026e-4712-a3e0-fd35565a3d89 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.676581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 884/1751] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:26.676581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f9b3c-026e-4712-a3e0-fd35565a3d89 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.684555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f9b3c-026e-4712-a3e0-fd35565a3d89 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.689207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6b213-ce1e-4c10-89f4-94db133e14b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.689832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6b213-ce1e-4c10-89f4-94db133e14b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.690116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6b213-ce1e-4c10-89f4-94db133e14b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.690405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6b213-ce1e-4c10-89f4-94db133e14b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.691114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413aba6-c596-4609-a811-773ed6af08b9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.691239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6b213-ce1e-4c10-89f4-94db133e14b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.693083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6b213-ce1e-4c10-89f4-94db133e14b5 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.695538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839830d-b7a0-490e-804f-27af530fc42d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.696249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839830d-b7a0-490e-804f-27af530fc42d service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.696600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839830d-b7a0-490e-804f-27af530fc42d service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.697201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839830d-b7a0-490e-804f-27af530fc42d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.706141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839830d-b7a0-490e-804f-27af530fc42d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.707672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839830d-b7a0-490e-804f-27af530fc42d service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.717587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 868/1752] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:26.719055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839830d-b7a0-490e-804f-27af530fc42d service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.719934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 885/1753] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:26.724614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6b213-ce1e-4c10-89f4-94db133e14b5 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.726715 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:26.738767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839830d-b7a0-490e-804f-27af530fc42d service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.801211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6b213-ce1e-4c10-89f4-94db133e14b5 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.806714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a4a21-9b6d-46ea-a1e1-9c4de9da84db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.807419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a4a21-9b6d-46ea-a1e1-9c4de9da84db admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.807706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a4a21-9b6d-46ea-a1e1-9c4de9da84db admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.807986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a4a21-9b6d-46ea-a1e1-9c4de9da84db admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.808673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a4a21-9b6d-46ea-a1e1-9c4de9da84db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.810604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a4a21-9b6d-46ea-a1e1-9c4de9da84db admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.810957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810282d1-78a2-46ff-9671-82f0dcdb62bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.811776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810282d1-78a2-46ff-9671-82f0dcdb62bd admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.812107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810282d1-78a2-46ff-9671-82f0dcdb62bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.812616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810282d1-78a2-46ff-9671-82f0dcdb62bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.813968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810282d1-78a2-46ff-9671-82f0dcdb62bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.816169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810282d1-78a2-46ff-9671-82f0dcdb62bd admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.828992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 869/1754] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:26.832843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 886/1755] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:50:26.833556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810282d1-78a2-46ff-9671-82f0dcdb62bd admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.836423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a4a21-9b6d-46ea-a1e1-9c4de9da84db admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.844183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810282d1-78a2-46ff-9671-82f0dcdb62bd admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.847849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a4a21-9b6d-46ea-a1e1-9c4de9da84db admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.849057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ef6f3-75eb-4eeb-94c9-fa57ca683262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.849946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ef6f3-75eb-4eeb-94c9-fa57ca683262 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.850232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ef6f3-75eb-4eeb-94c9-fa57ca683262 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.850537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ef6f3-75eb-4eeb-94c9-fa57ca683262 admin admin] PATH_INFO: `/v3/projects/a5425fc09f574ff7a4f8a2c89d8fcd9b/users/a9d24334a00a4df09326e20ea6dc1fa3/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.855231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce073863-03f3-4f9a-8a6e-b30fbc966ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.855901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce073863-03f3-4f9a-8a6e-b30fbc966ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.856189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce073863-03f3-4f9a-8a6e-b30fbc966ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.856466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce073863-03f3-4f9a-8a6e-b30fbc966ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.857198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce073863-03f3-4f9a-8a6e-b30fbc966ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.859438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce073863-03f3-4f9a-8a6e-b30fbc966ff8 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.869064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ef6f3-75eb-4eeb-94c9-fa57ca683262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5425fc09f574ff7a4f8a2c89d8fcd9b, user_id=a9d24334a00a4df09326e20ea6dc1fa3, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.870618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 887/1756] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:26.872368 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ef6f3-75eb-4eeb-94c9-fa57ca683262 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.874451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce073863-03f3-4f9a-8a6e-b30fbc966ff8 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:26.889096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce073863-03f3-4f9a-8a6e-b30fbc966ff8 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:26.892566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 870/1757] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:26 2022] PUT /identity/v3/projects/a5425fc09f574ff7a4f8a2c89d8fcd9b/users/a9d24334a00a4df09326e20ea6dc1fa3/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:26.895984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:26.906348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858d854-e7d9-4cde-a22d-140421c017d2 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.908132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858d854-e7d9-4cde-a22d-140421c017d2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.908443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858d854-e7d9-4cde-a22d-140421c017d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.939707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8858d854-e7d9-4cde-a22d-140421c017d2 None None] MFA Rules not processed for user `a9d24334a00a4df09326e20ea6dc1fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:26.943456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8858d854-e7d9-4cde-a22d-140421c017d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:26.983164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7eef8b-d819-4c56-8696-b63cd04a79f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:26.983883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eef8b-d819-4c56-8696-b63cd04a79f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:26.984176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eef8b-d819-4c56-8696-b63cd04a79f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:26.984464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eef8b-d819-4c56-8696-b63cd04a79f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:26.985193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7eef8b-d819-4c56-8696-b63cd04a79f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:26.987146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7eef8b-d819-4c56-8696-b63cd04a79f5 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:26.997958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 888/1758] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:27.001965 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7eef8b-d819-4c56-8696-b63cd04a79f5 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.012022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7eef8b-d819-4c56-8696-b63cd04a79f5 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.017321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7778c79-a218-4dda-9b3f-a8c4a33dae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.018520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7778c79-a218-4dda-9b3f-a8c4a33dae3f admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.018831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7778c79-a218-4dda-9b3f-a8c4a33dae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.019186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7778c79-a218-4dda-9b3f-a8c4a33dae3f admin admin] PATH_INFO: `/v3/projects/5a0c7a01f79a42219cab5cd2c84aef6a/users/9c2ba2d5e6db49ee805d9c03d865a822/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.032810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 871/1759] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:27.035397 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7778c79-a218-4dda-9b3f-a8c4a33dae3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a0c7a01f79a42219cab5cd2c84aef6a, user_id=9c2ba2d5e6db49ee805d9c03d865a822, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.037447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858d854-e7d9-4cde-a22d-140421c017d2 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.038818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7778c79-a218-4dda-9b3f-a8c4a33dae3f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.048452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858d854-e7d9-4cde-a22d-140421c017d2 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.053483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115b3f-e43e-4914-ab40-ac40641f23a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.054156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115b3f-e43e-4914-ab40-ac40641f23a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.054446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115b3f-e43e-4914-ab40-ac40641f23a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.055122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115b3f-e43e-4914-ab40-ac40641f23a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.055464 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115b3f-e43e-4914-ab40-ac40641f23a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.057432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115b3f-e43e-4914-ab40-ac40641f23a7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.059524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 889/1760] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:27 2022] PUT /identity/v3/projects/5a0c7a01f79a42219cab5cd2c84aef6a/users/9c2ba2d5e6db49ee805d9c03d865a822/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:27.069460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518e483-46e3-450c-b877-d8576322f60c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.069749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518e483-46e3-450c-b877-d8576322f60c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.070028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518e483-46e3-450c-b877-d8576322f60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.092897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 872/1761] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:27 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:50:27.094687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115b3f-e43e-4914-ab40-ac40641f23a7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.107631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-7518e483-46e3-450c-b877-d8576322f60c None None] MFA Rules not processed for user `9c2ba2d5e6db49ee805d9c03d865a822`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:27.109921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7518e483-46e3-450c-b877-d8576322f60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:27.180314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:27.205395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115b3f-e43e-4914-ab40-ac40641f23a7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.210845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7ba58-67f4-4cd5-82ae-c83033c61584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.211528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7ba58-67f4-4cd5-82ae-c83033c61584 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.212829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7ba58-67f4-4cd5-82ae-c83033c61584 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.213243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7ba58-67f4-4cd5-82ae-c83033c61584 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.213937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7ba58-67f4-4cd5-82ae-c83033c61584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.214766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 890/1762] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:27.215333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518e483-46e3-450c-b877-d8576322f60c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.217410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7ba58-67f4-4cd5-82ae-c83033c61584 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.218945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7518e483-46e3-450c-b877-d8576322f60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:27.234235 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518e483-46e3-450c-b877-d8576322f60c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.242357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.243610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.244002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.244360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] PATH_INFO: `/v3/projects/ed1fd50156824e9ba5f077dea1e56ab2` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.245817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1fd50156824e9ba5f077dea1e56ab2)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.248142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.259838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 873/1763] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:27 2022] POST /identity/v3/users => generated 302 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:27.259838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7ba58-67f4-4cd5-82ae-c83033c61584 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.277133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7ba58-67f4-4cd5-82ae-c83033c61584 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.282197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70ffed-e34e-44fd-beeb-fab9f41c80f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.282945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70ffed-e34e-44fd-beeb-fab9f41c80f7 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.283190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70ffed-e34e-44fd-beeb-fab9f41c80f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.283474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70ffed-e34e-44fd-beeb-fab9f41c80f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.284310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70ffed-e34e-44fd-beeb-fab9f41c80f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.286763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70ffed-e34e-44fd-beeb-fab9f41c80f7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.330066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 874/1764] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:27 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:27.330066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70ffed-e34e-44fd-beeb-fab9f41c80f7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.336269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1fd50156824e9ba5f077dea1e56ab2', 'request_id': 'req-239e62f5-8bff-4186-8df1-4594a53a9df8'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.344342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1fd50156824e9ba5f077dea1e56ab2', 'request_id': 'req-239e62f5-8bff-4186-8df1-4594a53a9df8'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.353617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1fd50156824e9ba5f077dea1e56ab2', 'request_id': 'req-239e62f5-8bff-4186-8df1-4594a53a9df8'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.356599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:27.370759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] The token cache is being invalidate because project ed1fd50156824e9ba5f077dea1e56ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:27.371217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.372089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.374923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 891/1765] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:27 2022] DELETE /identity/v3/projects/ed1fd50156824e9ba5f077dea1e56ab2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:27.375912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.377648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:27.401048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70ffed-e34e-44fd-beeb-fab9f41c80f7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.411637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa7ebc-c421-471b-be92-aa0049158a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.412556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa7ebc-c421-471b-be92-aa0049158a77 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.413008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa7ebc-c421-471b-be92-aa0049158a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.413418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa7ebc-c421-471b-be92-aa0049158a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.414343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa7ebc-c421-471b-be92-aa0049158a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.416046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa7ebc-c421-471b-be92-aa0049158a77 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.430081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 875/1766] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:27.430887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa7ebc-c421-471b-be92-aa0049158a77 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.435657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-aaaa7ebc-c421-471b-be92-aa0049158a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:27.453733 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa7ebc-c421-471b-be92-aa0049158a77 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.459095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.460092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.460503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.460807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] PATH_INFO: `/v3/users/78e91027e2404b5b9d2b14b509d35a77` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.462164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e91027e2404b5b9d2b14b509d35a77)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.464255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.476081 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e62f5-8bff-4186-8df1-4594a53a9df8 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.482029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287edd-e96a-4422-bbe3-206c6a448214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.482708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287edd-e96a-4422-bbe3-206c6a448214 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.483024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287edd-e96a-4422-bbe3-206c6a448214 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.483304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287edd-e96a-4422-bbe3-206c6a448214 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.484034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82287edd-e96a-4422-bbe3-206c6a448214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.485965 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82287edd-e96a-4422-bbe3-206c6a448214 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.511362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 892/1767] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:27 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:27.511362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287edd-e96a-4422-bbe3-206c6a448214 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.511683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-82287edd-e96a-4422-bbe3-206c6a448214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:27.519123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e91027e2404b5b9d2b14b509d35a77'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.526132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287edd-e96a-4422-bbe3-206c6a448214 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.528168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e91027e2404b5b9d2b14b509d35a77'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.531527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84815bd9-fb92-4e75-aee4-3de6df8b19e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.532499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84815bd9-fb92-4e75-aee4-3de6df8b19e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.533514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84815bd9-fb92-4e75-aee4-3de6df8b19e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.533514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84815bd9-fb92-4e75-aee4-3de6df8b19e7 admin admin] PATH_INFO: `/v3/projects/3e449c795d1a46caa00f0dd7b4cc66af/users/e74cd68a5ff942559a771da8159a75ea/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.536798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e91027e2404b5b9d2b14b509d35a77'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.538040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e91027e2404b5b9d2b14b509d35a77'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.538993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e91027e2404b5b9d2b14b509d35a77'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:27.554054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84815bd9-fb92-4e75-aee4-3de6df8b19e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e449c795d1a46caa00f0dd7b4cc66af, user_id=e74cd68a5ff942559a771da8159a75ea, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.556795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84815bd9-fb92-4e75-aee4-3de6df8b19e7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.558791 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 876/1768] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:27 2022] DELETE /identity/v3/users/78e91027e2404b5b9d2b14b509d35a77 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:27.559532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.560966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:27.573426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 893/1769] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:27 2022] PUT /identity/v3/projects/3e449c795d1a46caa00f0dd7b4cc66af/users/e74cd68a5ff942559a771da8159a75ea/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:27.577274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84815bd9-fb92-4e75-aee4-3de6df8b19e7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.581419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-84815bd9-fb92-4e75-aee4-3de6df8b19e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:27.591965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:27.652110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc407f-038d-42f3-93b2-65e13dc33564 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.672014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84815bd9-fb92-4e75-aee4-3de6df8b19e7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.678386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df6369-e6bb-4f70-9375-cb7d7d55ca5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.679014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df6369-e6bb-4f70-9375-cb7d7d55ca5c admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.679136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df6369-e6bb-4f70-9375-cb7d7d55ca5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.679736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df6369-e6bb-4f70-9375-cb7d7d55ca5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.680148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df6369-e6bb-4f70-9375-cb7d7d55ca5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.682394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df6369-e6bb-4f70-9375-cb7d7d55ca5c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.698904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 894/1770] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:27.701250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df6369-e6bb-4f70-9375-cb7d7d55ca5c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.713694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df6369-e6bb-4f70-9375-cb7d7d55ca5c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.720266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80a3bd-e612-4cb6-9016-0f677f2aca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.721281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80a3bd-e612-4cb6-9016-0f677f2aca7a admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.721564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80a3bd-e612-4cb6-9016-0f677f2aca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.721838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80a3bd-e612-4cb6-9016-0f677f2aca7a admin admin] PATH_INFO: `/v3/projects/3e449c795d1a46caa00f0dd7b4cc66af/users/e74cd68a5ff942559a771da8159a75ea/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.724525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80a3bd-e612-4cb6-9016-0f677f2aca7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e449c795d1a46caa00f0dd7b4cc66af, user_id=e74cd68a5ff942559a771da8159a75ea, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.724665 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c38a5-193c-4922-8c4d-b0cf6c5fff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.725654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c38a5-193c-4922-8c4d-b0cf6c5fff21 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.726009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c38a5-193c-4922-8c4d-b0cf6c5fff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.726403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c38a5-193c-4922-8c4d-b0cf6c5fff21 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.727244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c38a5-193c-4922-8c4d-b0cf6c5fff21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.728037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80a3bd-e612-4cb6-9016-0f677f2aca7a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.729077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c38a5-193c-4922-8c4d-b0cf6c5fff21 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.741156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 877/1771] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:27.745196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c38a5-193c-4922-8c4d-b0cf6c5fff21 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.746804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 895/1772] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:27 2022] PUT /identity/v3/projects/3e449c795d1a46caa00f0dd7b4cc66af/users/e74cd68a5ff942559a771da8159a75ea/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:27.754544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f803e21-5982-4e17-af16-1cf73ec4f1e8 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.754883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f803e21-5982-4e17-af16-1cf73ec4f1e8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.755166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f803e21-5982-4e17-af16-1cf73ec4f1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.763765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c38a5-193c-4922-8c4d-b0cf6c5fff21 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.783179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-3f803e21-5982-4e17-af16-1cf73ec4f1e8 None None] MFA Rules not processed for user `e74cd68a5ff942559a771da8159a75ea`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:27.784398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3f803e21-5982-4e17-af16-1cf73ec4f1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:27.821502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:27.839962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023feb6-db1d-4212-8988-44fe210ae849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.841184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023feb6-db1d-4212-8988-44fe210ae849 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.841469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023feb6-db1d-4212-8988-44fe210ae849 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.841748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023feb6-db1d-4212-8988-44fe210ae849 admin admin] PATH_INFO: `/v3/projects/c1527654fe3649099f96a6301bb14838/users/0069c5f233ac4d5daab1f6d8632979cb/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.860389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023feb6-db1d-4212-8988-44fe210ae849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1527654fe3649099f96a6301bb14838, user_id=0069c5f233ac4d5daab1f6d8632979cb, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.863024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023feb6-db1d-4212-8988-44fe210ae849 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.872259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 896/1773] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:27.883794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f803e21-5982-4e17-af16-1cf73ec4f1e8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.883943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 878/1774] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:27 2022] PUT /identity/v3/projects/c1527654fe3649099f96a6301bb14838/users/0069c5f233ac4d5daab1f6d8632979cb/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:27.887962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3f803e21-5982-4e17-af16-1cf73ec4f1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:27.888560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023feb6-db1d-4212-8988-44fe210ae849 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.947444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:27.967739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f803e21-5982-4e17-af16-1cf73ec4f1e8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.972489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023feb6-db1d-4212-8988-44fe210ae849 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:27.977387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e10b0-77e6-4e9e-b5e5-32816e22d2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:27.978695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e10b0-77e6-4e9e-b5e5-32816e22d2fc admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:27.978695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e10b0-77e6-4e9e-b5e5-32816e22d2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:27.979014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e10b0-77e6-4e9e-b5e5-32816e22d2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:27.979509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e10b0-77e6-4e9e-b5e5-32816e22d2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:27.981226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e10b0-77e6-4e9e-b5e5-32816e22d2fc admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:27.996467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 879/1775] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:27.996467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e10b0-77e6-4e9e-b5e5-32816e22d2fc admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:27.998088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c76e10b0-77e6-4e9e-b5e5-32816e22d2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:28.012941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e10b0-77e6-4e9e-b5e5-32816e22d2fc admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.021268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.022085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.022419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.023692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] PATH_INFO: `/v3/projects/68fb14a59e0c4949ad7259fb7eb4b3ac` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.024595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68fb14a59e0c4949ad7259fb7eb4b3ac)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.026612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.040668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc5f54-1e52-4a6e-b6cd-135769dc8668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.041776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc5f54-1e52-4a6e-b6cd-135769dc8668 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.042241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc5f54-1e52-4a6e-b6cd-135769dc8668 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.042757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc5f54-1e52-4a6e-b6cd-135769dc8668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.088870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68fb14a59e0c4949ad7259fb7eb4b3ac', 'request_id': 'req-636e7284-f1a6-444b-a262-ab89c9422149'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:28.096794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68fb14a59e0c4949ad7259fb7eb4b3ac', 'request_id': 'req-636e7284-f1a6-444b-a262-ab89c9422149'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:28.104776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68fb14a59e0c4949ad7259fb7eb4b3ac', 'request_id': 'req-636e7284-f1a6-444b-a262-ab89c9422149'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:28.114348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc5f54-1e52-4a6e-b6cd-135769dc8668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.117175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc5f54-1e52-4a6e-b6cd-135769dc8668 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.120227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] The token cache is being invalidate because project 68fb14a59e0c4949ad7259fb7eb4b3ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:28.120506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:28.121275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:28.123251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 880/1776] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:27 2022] DELETE /identity/v3/projects/68fb14a59e0c4949ad7259fb7eb4b3ac => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:28.127570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:28.127570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:28.131331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 897/1777] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:50:28.132510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf1bc4-40c0-4c40-bb65-e9f2be3c8569 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.132800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf1bc4-40c0-4c40-bb65-e9f2be3c8569 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.133085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf1bc4-40c0-4c40-bb65-e9f2be3c8569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.150891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-d8bf1bc4-40c0-4c40-bb65-e9f2be3c8569 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:28.152309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d8bf1bc4-40c0-4c40-bb65-e9f2be3c8569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:28.188812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:28.214178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7284-f1a6-444b-a262-ab89c9422149 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.221892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f60460-3057-45cb-bb12-8abfd857bbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.222847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f60460-3057-45cb-bb12-8abfd857bbca admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.223116 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f60460-3057-45cb-bb12-8abfd857bbca admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.223437 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f60460-3057-45cb-bb12-8abfd857bbca admin admin] PATH_INFO: `/v3/projects/c1527654fe3649099f96a6301bb14838/users/0069c5f233ac4d5daab1f6d8632979cb/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.225491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f60460-3057-45cb-bb12-8abfd857bbca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1527654fe3649099f96a6301bb14838, user_id=0069c5f233ac4d5daab1f6d8632979cb, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.233047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 898/1778] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:28.233047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf1bc4-40c0-4c40-bb65-e9f2be3c8569 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:28.233320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f60460-3057-45cb-bb12-8abfd857bbca admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.234946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d8bf1bc4-40c0-4c40-bb65-e9f2be3c8569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:28.244587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 881/1779] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:28 2022] PUT /identity/v3/projects/c1527654fe3649099f96a6301bb14838/users/0069c5f233ac4d5daab1f6d8632979cb/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:28.255255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6c35a-0a4c-4b2c-8569-09c8ccd41e11 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.255554 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6c35a-0a4c-4b2c-8569-09c8ccd41e11 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.255838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6c35a-0a4c-4b2c-8569-09c8ccd41e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.277060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d0f6c35a-0a4c-4b2c-8569-09c8ccd41e11 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:28.278360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d0f6c35a-0a4c-4b2c-8569-09c8ccd41e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:28.302655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf1bc4-40c0-4c40-bb65-e9f2be3c8569 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.351146 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:28.363943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 882/1780] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:28.364910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6146cb-9f3b-4bee-a44b-29c1b98ce857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.365575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6146cb-9f3b-4bee-a44b-29c1b98ce857 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.365860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6146cb-9f3b-4bee-a44b-29c1b98ce857 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.366145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6146cb-9f3b-4bee-a44b-29c1b98ce857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.368319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad2648-b3cc-4eda-97ca-4c97bb677582 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.368614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad2648-b3cc-4eda-97ca-4c97bb677582 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.368696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7f6146cb-9f3b-4bee-a44b-29c1b98ce857 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:28.368894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad2648-b3cc-4eda-97ca-4c97bb677582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.395769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-69ad2648-b3cc-4eda-97ca-4c97bb677582 None None] MFA Rules not processed for user `0069c5f233ac4d5daab1f6d8632979cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:28.396978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-69ad2648-b3cc-4eda-97ca-4c97bb677582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:28.436498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6146cb-9f3b-4bee-a44b-29c1b98ce857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.442412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6146cb-9f3b-4bee-a44b-29c1b98ce857 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.469056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 899/1781] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:28.470210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6146cb-9f3b-4bee-a44b-29c1b98ce857 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:28.486194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 883/1782] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:28.488121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6146cb-9f3b-4bee-a44b-29c1b98ce857 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.495995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.497681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.498499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.498951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.500051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:50:28.500582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.503133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.524110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 900/1783] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:50:28.526175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad2648-b3cc-4eda-97ca-4c97bb677582 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:28.539591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad2648-b3cc-4eda-97ca-4c97bb677582 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.549200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b3e45-d45d-432f-acb3-37ec9f74f5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.551484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b3e45-d45d-432f-acb3-37ec9f74f5ed admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.551784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b3e45-d45d-432f-acb3-37ec9f74f5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.552072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b3e45-d45d-432f-acb3-37ec9f74f5ed admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.552779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b3e45-d45d-432f-acb3-37ec9f74f5ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.554818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b3e45-d45d-432f-acb3-37ec9f74f5ed admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.557138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:28.559061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:28.573727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929fad0f-c03a-4960-9c06-153ee550e5c7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.579451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b0ec0-8303-4aa1-93ca-6b9ae50ee24b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.580161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b0ec0-8303-4aa1-93ca-6b9ae50ee24b service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.580444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b0ec0-8303-4aa1-93ca-6b9ae50ee24b service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.580723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b0ec0-8303-4aa1-93ca-6b9ae50ee24b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.593121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 884/1784] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:28 2022] POST /identity/v3/projects => generated 397 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:28.594344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b0ec0-8303-4aa1-93ca-6b9ae50ee24b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.595060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b0ec0-8303-4aa1-93ca-6b9ae50ee24b service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.599100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b3e45-d45d-432f-acb3-37ec9f74f5ed admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:28.609626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 901/1785] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:28.699484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b3e45-d45d-432f-acb3-37ec9f74f5ed admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.707037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89554d-52a8-477b-9a6c-5b29e6f23ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.707817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89554d-52a8-477b-9a6c-5b29e6f23ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.708105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89554d-52a8-477b-9a6c-5b29e6f23ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.708435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89554d-52a8-477b-9a6c-5b29e6f23ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.709147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89554d-52a8-477b-9a6c-5b29e6f23ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.711357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89554d-52a8-477b-9a6c-5b29e6f23ad0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.740035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 885/1786] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:28 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:28.744361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b0ec0-8303-4aa1-93ca-6b9ae50ee24b service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:28.755252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b0ec0-8303-4aa1-93ca-6b9ae50ee24b service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.760784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54fe6b-d814-47fd-9a56-64e6dc9adc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.761506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54fe6b-d814-47fd-9a56-64e6dc9adc86 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.761791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54fe6b-d814-47fd-9a56-64e6dc9adc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.762068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54fe6b-d814-47fd-9a56-64e6dc9adc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.763655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54fe6b-d814-47fd-9a56-64e6dc9adc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.766427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54fe6b-d814-47fd-9a56-64e6dc9adc86 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.788936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 902/1787] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:28.793142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89554d-52a8-477b-9a6c-5b29e6f23ad0 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:28.807939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89554d-52a8-477b-9a6c-5b29e6f23ad0 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.814339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338daad-7aea-4794-abf5-d2e4c61c6ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.815812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338daad-7aea-4794-abf5-d2e4c61c6ebd admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.816128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338daad-7aea-4794-abf5-d2e4c61c6ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.816563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338daad-7aea-4794-abf5-d2e4c61c6ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.817796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338daad-7aea-4794-abf5-d2e4c61c6ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.819835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338daad-7aea-4794-abf5-d2e4c61c6ebd admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.851279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 886/1788] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:28.852065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54fe6b-d814-47fd-9a56-64e6dc9adc86 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:28.862962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:28.869218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54fe6b-d814-47fd-9a56-64e6dc9adc86 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:28.878223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ec9b2-117a-4ea7-937c-6ccc205d830e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:28.879847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ec9b2-117a-4ea7-937c-6ccc205d830e admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.880314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ec9b2-117a-4ea7-937c-6ccc205d830e admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.880871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ec9b2-117a-4ea7-937c-6ccc205d830e admin admin] PATH_INFO: `/v3/projects/3e6cd256adae4dec948b931f9e773f32/users/11437e5915ae466687a2ef6f8ae551bc/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.900635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ec9b2-117a-4ea7-937c-6ccc205d830e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e6cd256adae4dec948b931f9e773f32, user_id=11437e5915ae466687a2ef6f8ae551bc, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:28.903571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ec9b2-117a-4ea7-937c-6ccc205d830e admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:28.940641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc5fd5-f5b9-4bae-b56c-1d5fdb195804 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:28.941298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc5fd5-f5b9-4bae-b56c-1d5fdb195804 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:28.941711 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc5fd5-f5b9-4bae-b56c-1d5fdb195804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:28.942104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 903/1789] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:28 2022] PUT /identity/v3/projects/3e6cd256adae4dec948b931f9e773f32/users/11437e5915ae466687a2ef6f8ae551bc/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:28.951584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ec9b2-117a-4ea7-937c-6ccc205d830e admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:29.017619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ddbc5fd5-f5b9-4bae-b56c-1d5fdb195804 None None] MFA Rules not processed for user `11437e5915ae466687a2ef6f8ae551bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:29.019051 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ddbc5fd5-f5b9-4bae-b56c-1d5fdb195804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:29.025806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ec9b2-117a-4ea7-937c-6ccc205d830e admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:29.049461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18416e2-76d1-4397-978f-1209c5f20664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:29.050221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18416e2-76d1-4397-978f-1209c5f20664 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.050515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18416e2-76d1-4397-978f-1209c5f20664 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.050859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18416e2-76d1-4397-978f-1209c5f20664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.053757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d18416e2-76d1-4397-978f-1209c5f20664 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:29.055163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:29.101237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 887/1790] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:29.105363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc5fd5-f5b9-4bae-b56c-1d5fdb195804 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:29.132999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18416e2-76d1-4397-978f-1209c5f20664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:29.134639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18416e2-76d1-4397-978f-1209c5f20664 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:29.151535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 904/1791] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:50:29.180762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc5fd5-f5b9-4bae-b56c-1d5fdb195804 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:29.185440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e282fa9-ac19-447c-8681-31be91651b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:29.186056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e282fa9-ac19-447c-8681-31be91651b39 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.186371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e282fa9-ac19-447c-8681-31be91651b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.186654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e282fa9-ac19-447c-8681-31be91651b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.187919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e282fa9-ac19-447c-8681-31be91651b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:29.189804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e282fa9-ac19-447c-8681-31be91651b39 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:29.216860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 888/1792] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:29 2022] POST /identity/v3/projects => generated 399 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:50:29.220955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18416e2-76d1-4397-978f-1209c5f20664 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:29.232065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18416e2-76d1-4397-978f-1209c5f20664 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:29.297228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92614455-dbc8-4c5e-ba73-c8d1fdebf276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:29.297921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92614455-dbc8-4c5e-ba73-c8d1fdebf276 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.298206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92614455-dbc8-4c5e-ba73-c8d1fdebf276 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.298482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92614455-dbc8-4c5e-ba73-c8d1fdebf276 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.299215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92614455-dbc8-4c5e-ba73-c8d1fdebf276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:29.301135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92614455-dbc8-4c5e-ba73-c8d1fdebf276 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:29.322493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 905/1793] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:29 2022] POST /identity/v3/users => generated 306 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:29.326572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e282fa9-ac19-447c-8681-31be91651b39 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:29.337388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e282fa9-ac19-447c-8681-31be91651b39 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:29.341998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18b0ad-c154-4262-a4cf-cfae0a8acaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:29.342660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18b0ad-c154-4262-a4cf-cfae0a8acaaa admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.342982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18b0ad-c154-4262-a4cf-cfae0a8acaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.343292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18b0ad-c154-4262-a4cf-cfae0a8acaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.344018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18b0ad-c154-4262-a4cf-cfae0a8acaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:29.345662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18b0ad-c154-4262-a4cf-cfae0a8acaaa admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:29.356288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 889/1794] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:29.360903 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92614455-dbc8-4c5e-ba73-c8d1fdebf276 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:29.371452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92614455-dbc8-4c5e-ba73-c8d1fdebf276 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:29.378639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86761b9-9574-467f-a8a6-6a054e3b4e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:29.379630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86761b9-9574-467f-a8a6-6a054e3b4e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.379919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86761b9-9574-467f-a8a6-6a054e3b4e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.380199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86761b9-9574-467f-a8a6-6a054e3b4e05 admin admin] PATH_INFO: `/v3/projects/09462e301a6d4a9db49be1762803c407/users/f280008a3a7c4002acedb29627af4886/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.395335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86761b9-9574-467f-a8a6-6a054e3b4e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09462e301a6d4a9db49be1762803c407, user_id=f280008a3a7c4002acedb29627af4886, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:29.397991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86761b9-9574-467f-a8a6-6a054e3b4e05 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:29.409175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:29.415626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 906/1795] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:29 2022] PUT /identity/v3/projects/09462e301a6d4a9db49be1762803c407/users/f280008a3a7c4002acedb29627af4886/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:29.420036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18b0ad-c154-4262-a4cf-cfae0a8acaaa admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:29.430956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18b0ad-c154-4262-a4cf-cfae0a8acaaa admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:29.498392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a3fcd-a72a-4b73-a905-f7fae727a8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:29.499113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a3fcd-a72a-4b73-a905-f7fae727a8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.499971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a3fcd-a72a-4b73-a905-f7fae727a8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.500252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a3fcd-a72a-4b73-a905-f7fae727a8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.500988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a3fcd-a72a-4b73-a905-f7fae727a8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:29.502590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a3fcd-a72a-4b73-a905-f7fae727a8f4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:29.516002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 890/1796] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:29.517645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86761b9-9574-467f-a8a6-6a054e3b4e05 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:29.529555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86761b9-9574-467f-a8a6-6a054e3b4e05 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:29.534716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2033c-ca8b-461a-ac5b-616c5162f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:29.535751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2033c-ca8b-461a-ac5b-616c5162f709 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.536408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2033c-ca8b-461a-ac5b-616c5162f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.537040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2033c-ca8b-461a-ac5b-616c5162f709 admin admin] PATH_INFO: `/v3/projects/09462e301a6d4a9db49be1762803c407/users/f280008a3a7c4002acedb29627af4886/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.539169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2033c-ca8b-461a-ac5b-616c5162f709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09462e301a6d4a9db49be1762803c407, user_id=f280008a3a7c4002acedb29627af4886, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:29.542493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2033c-ca8b-461a-ac5b-616c5162f709 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:29.562989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 907/1797] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:29 2022] PUT /identity/v3/projects/09462e301a6d4a9db49be1762803c407/users/f280008a3a7c4002acedb29627af4886/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:29.574037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42d262-50bc-4b8f-82bc-49fc2437847f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.574329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42d262-50bc-4b8f-82bc-49fc2437847f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.574605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42d262-50bc-4b8f-82bc-49fc2437847f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.603158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8b42d262-50bc-4b8f-82bc-49fc2437847f None None] MFA Rules not processed for user `f280008a3a7c4002acedb29627af4886`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:29.604383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8b42d262-50bc-4b8f-82bc-49fc2437847f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:29.634659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:29.678683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 891/1798] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:29.688995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2033c-ca8b-461a-ac5b-616c5162f709 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:29.743797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2033c-ca8b-461a-ac5b-616c5162f709 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:29.747477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932784c-f5ca-4cb2-941c-3c4765fbd077 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:29.748124 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932784c-f5ca-4cb2-941c-3c4765fbd077 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.748407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932784c-f5ca-4cb2-941c-3c4765fbd077 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.748686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932784c-f5ca-4cb2-941c-3c4765fbd077 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.757831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932784c-f5ca-4cb2-941c-3c4765fbd077 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:29.759307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932784c-f5ca-4cb2-941c-3c4765fbd077 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:29.768744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 908/1799] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:29.850197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42d262-50bc-4b8f-82bc-49fc2437847f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:29.860162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42d262-50bc-4b8f-82bc-49fc2437847f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:29.863886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5433b9-cab7-4127-b47e-30f3d7f3d647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:29.864531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5433b9-cab7-4127-b47e-30f3d7f3d647 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:29.864817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5433b9-cab7-4127-b47e-30f3d7f3d647 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:29.865091 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5433b9-cab7-4127-b47e-30f3d7f3d647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:29.925373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5433b9-cab7-4127-b47e-30f3d7f3d647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:29.926876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5433b9-cab7-4127-b47e-30f3d7f3d647 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:29.935962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 892/1800] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:34.120419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932784c-f5ca-4cb2-941c-3c4765fbd077 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:34.214306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932784c-f5ca-4cb2-941c-3c4765fbd077 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:34.220404 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:34.221726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:34.222278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:34.222787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] PATH_INFO: `/v3/users/9c2ba2d5e6db49ee805d9c03d865a822` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:34.224346 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c2ba2d5e6db49ee805d9c03d865a822)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:34.226522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:34.293812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c2ba2d5e6db49ee805d9c03d865a822'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:34.304581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c2ba2d5e6db49ee805d9c03d865a822'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:34.320485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c2ba2d5e6db49ee805d9c03d865a822'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:34.321578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c2ba2d5e6db49ee805d9c03d865a822'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:34.339163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c2ba2d5e6db49ee805d9c03d865a822'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:34.353651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 909/1801] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:34 2022] DELETE /identity/v3/users/9c2ba2d5e6db49ee805d9c03d865a822 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:34.355351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865fa23-0da6-4bdd-b443-7fcd9a468d86 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:34.355796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865fa23-0da6-4bdd-b443-7fcd9a468d86 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:34.355996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865fa23-0da6-4bdd-b443-7fcd9a468d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:34.381760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-4865fa23-0da6-4bdd-b443-7fcd9a468d86 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:34.383013 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4865fa23-0da6-4bdd-b443-7fcd9a468d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:34.478561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 893/1802] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:34.491092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:34.492719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:34.504795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:34.573747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24bbe4-8e1b-4be9-a0b4-776f4268ea41 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:34.578015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f84b4-dd17-412b-9c80-cfdcab52fada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:34.578689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f84b4-dd17-412b-9c80-cfdcab52fada service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:34.579144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f84b4-dd17-412b-9c80-cfdcab52fada service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:34.579432 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f84b4-dd17-412b-9c80-cfdcab52fada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:34.589583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f84b4-dd17-412b-9c80-cfdcab52fada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:34.590586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f84b4-dd17-412b-9c80-cfdcab52fada service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:34.601607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 910/1803] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:34.887260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865fa23-0da6-4bdd-b443-7fcd9a468d86 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:34.891159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4865fa23-0da6-4bdd-b443-7fcd9a468d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:34.921032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865fa23-0da6-4bdd-b443-7fcd9a468d86 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:34.927269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:34.928310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:34.929288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:34.929583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] PATH_INFO: `/v3/projects/5a0c7a01f79a42219cab5cd2c84aef6a` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:34.930847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a0c7a01f79a42219cab5cd2c84aef6a)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:34.932758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:34.993134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a0c7a01f79a42219cab5cd2c84aef6a', 'request_id': 'req-3b46af09-1999-4bdd-ab1d-a429ac9daf86'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:34.999818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a0c7a01f79a42219cab5cd2c84aef6a', 'request_id': 'req-3b46af09-1999-4bdd-ab1d-a429ac9daf86'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.004393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:35.009580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a0c7a01f79a42219cab5cd2c84aef6a', 'request_id': 'req-3b46af09-1999-4bdd-ab1d-a429ac9daf86'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.026428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] The token cache is being invalidate because project 5a0c7a01f79a42219cab5cd2c84aef6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:35.026750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.027476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.029209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 894/1804] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:34 2022] DELETE /identity/v3/projects/5a0c7a01f79a42219cab5cd2c84aef6a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:35.033624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f84b4-dd17-412b-9c80-cfdcab52fada service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:35.044020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f84b4-dd17-412b-9c80-cfdcab52fada service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:35.112975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:35.114116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:35.114407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:35.114957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] PATH_INFO: `/v3/users/0069c5f233ac4d5daab1f6d8632979cb` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:35.118895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0069c5f233ac4d5daab1f6d8632979cb)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:35.121301 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:35.177232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0069c5f233ac4d5daab1f6d8632979cb'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.183767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0069c5f233ac4d5daab1f6d8632979cb'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.191751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0069c5f233ac4d5daab1f6d8632979cb'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.192466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0069c5f233ac4d5daab1f6d8632979cb'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.205615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c9701b5-9a0c-4cd8-9fc4-cfe118093307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0069c5f233ac4d5daab1f6d8632979cb'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.211657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 911/1805] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:35 2022] DELETE /identity/v3/users/0069c5f233ac4d5daab1f6d8632979cb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:35.581270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:35.583286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:35.606016 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46af09-1999-4bdd-ab1d-a429ac9daf86 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:35.680328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:35.681681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:35.682155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:35.682773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] PATH_INFO: `/v3/projects/c1527654fe3649099f96a6301bb14838` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:35.684827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1527654fe3649099f96a6301bb14838)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:35.687673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:35.764141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1527654fe3649099f96a6301bb14838', 'request_id': 'req-32134b31-ac54-4191-be46-97b6d74c6d2a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.773758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1527654fe3649099f96a6301bb14838', 'request_id': 'req-32134b31-ac54-4191-be46-97b6d74c6d2a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.787668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1527654fe3649099f96a6301bb14838', 'request_id': 'req-32134b31-ac54-4191-be46-97b6d74c6d2a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.802494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] The token cache is being invalidate because project c1527654fe3649099f96a6301bb14838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:35.802884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.803702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-32134b31-ac54-4191-be46-97b6d74c6d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:35.805384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 895/1806] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:35 2022] DELETE /identity/v3/projects/c1527654fe3649099f96a6301bb14838 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:35.810041 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315ea0f-97a5-4691-af8f-a393910df0dd None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:35.810406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315ea0f-97a5-4691-af8f-a393910df0dd None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:35.810798 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315ea0f-97a5-4691-af8f-a393910df0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:35.832115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2315ea0f-97a5-4691-af8f-a393910df0dd None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:35.833385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2315ea0f-97a5-4691-af8f-a393910df0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:35.858759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:35.894072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 912/1807] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:35.902030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b9220-f7e7-4bad-b452-b423698029d9 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:35.902319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b9220-f7e7-4bad-b452-b423698029d9 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:35.902599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b9220-f7e7-4bad-b452-b423698029d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:35.920642 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-527b9220-f7e7-4bad-b452-b423698029d9 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:35.921849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-527b9220-f7e7-4bad-b452-b423698029d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:35.929495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 896/1808] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:35.933591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315ea0f-97a5-4691-af8f-a393910df0dd None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:35.943831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315ea0f-97a5-4691-af8f-a393910df0dd None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:35.948497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b604cf-7b5a-4fda-bd9b-bd46ef3bbb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:35.949146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b604cf-7b5a-4fda-bd9b-bd46ef3bbb47 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:35.949426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b604cf-7b5a-4fda-bd9b-bd46ef3bbb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:35.949700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b604cf-7b5a-4fda-bd9b-bd46ef3bbb47 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:35.950440 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b604cf-7b5a-4fda-bd9b-bd46ef3bbb47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:50:35.950761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b604cf-7b5a-4fda-bd9b-bd46ef3bbb47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:35.953058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b604cf-7b5a-4fda-bd9b-bd46ef3bbb47 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:35.964453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 913/1809] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:50:35.968508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b9220-f7e7-4bad-b452-b423698029d9 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:35.977499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b9220-f7e7-4bad-b452-b423698029d9 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:35.982027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2e405-d671-4b02-93a5-f25db35a4509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:35.982666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2e405-d671-4b02-93a5-f25db35a4509 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:35.983356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2e405-d671-4b02-93a5-f25db35a4509 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:35.983356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2e405-d671-4b02-93a5-f25db35a4509 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:35.984461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2e405-d671-4b02-93a5-f25db35a4509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:35.986337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2e405-d671-4b02-93a5-f25db35a4509 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.011935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 897/1810] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:35 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:36.015945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b604cf-7b5a-4fda-bd9b-bd46ef3bbb47 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.025854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b604cf-7b5a-4fda-bd9b-bd46ef3bbb47 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.082624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464acf5-2da9-4e1e-9694-5d3cfc8aa6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.083329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464acf5-2da9-4e1e-9694-5d3cfc8aa6cf admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.083613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464acf5-2da9-4e1e-9694-5d3cfc8aa6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.083893 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464acf5-2da9-4e1e-9694-5d3cfc8aa6cf admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.084575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464acf5-2da9-4e1e-9694-5d3cfc8aa6cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.086489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464acf5-2da9-4e1e-9694-5d3cfc8aa6cf admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.106576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 914/1811] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:36 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:50:36.111308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2e405-d671-4b02-93a5-f25db35a4509 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.122672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2e405-d671-4b02-93a5-f25db35a4509 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.128044 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178a1e6-3c6c-41f8-aa04-249bd9e77848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.128714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178a1e6-3c6c-41f8-aa04-249bd9e77848 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.129008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178a1e6-3c6c-41f8-aa04-249bd9e77848 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.129307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178a1e6-3c6c-41f8-aa04-249bd9e77848 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.130019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178a1e6-3c6c-41f8-aa04-249bd9e77848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.131653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178a1e6-3c6c-41f8-aa04-249bd9e77848 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.143345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 898/1812] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:36.147532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464acf5-2da9-4e1e-9694-5d3cfc8aa6cf admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.157885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464acf5-2da9-4e1e-9694-5d3cfc8aa6cf admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.162861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04b71a-f709-42fe-b6d1-9d902bdedf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.163525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b71a-f709-42fe-b6d1-9d902bdedf43 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.163811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b71a-f709-42fe-b6d1-9d902bdedf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.164091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b71a-f709-42fe-b6d1-9d902bdedf43 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.164856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04b71a-f709-42fe-b6d1-9d902bdedf43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.166448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04b71a-f709-42fe-b6d1-9d902bdedf43 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.178659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 915/1813] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:36.182087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178a1e6-3c6c-41f8-aa04-249bd9e77848 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.192529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178a1e6-3c6c-41f8-aa04-249bd9e77848 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.197188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9efd1-3130-4a85-b19f-a452e3641d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.198078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9efd1-3130-4a85-b19f-a452e3641d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.198359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9efd1-3130-4a85-b19f-a452e3641d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.198636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9efd1-3130-4a85-b19f-a452e3641d00 admin admin] PATH_INFO: `/v3/projects/c09be2db83974b65bc31b96f31e7587b/users/7210b91378e84e5887d867a0502104eb/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.217039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9efd1-3130-4a85-b19f-a452e3641d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09be2db83974b65bc31b96f31e7587b, user_id=7210b91378e84e5887d867a0502104eb, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.220671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9efd1-3130-4a85-b19f-a452e3641d00 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.240265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 899/1814] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:36 2022] PUT /identity/v3/projects/c09be2db83974b65bc31b96f31e7587b/users/7210b91378e84e5887d867a0502104eb/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:36.251199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca024a58-1ba5-40b6-afee-2f563ac3eef3 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.251486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca024a58-1ba5-40b6-afee-2f563ac3eef3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.251782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca024a58-1ba5-40b6-afee-2f563ac3eef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.278484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ca024a58-1ba5-40b6-afee-2f563ac3eef3 None None] MFA Rules not processed for user `7210b91378e84e5887d867a0502104eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:36.279654 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ca024a58-1ba5-40b6-afee-2f563ac3eef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:36.343596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 916/1815] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:36.348196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9efd1-3130-4a85-b19f-a452e3641d00 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.361947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:36.411742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9efd1-3130-4a85-b19f-a452e3641d00 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.416321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6ffa7-855a-4e33-88eb-4749f09035f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.416973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6ffa7-855a-4e33-88eb-4749f09035f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.417256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6ffa7-855a-4e33-88eb-4749f09035f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.417536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6ffa7-855a-4e33-88eb-4749f09035f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.418234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6ffa7-855a-4e33-88eb-4749f09035f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.420148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6ffa7-855a-4e33-88eb-4749f09035f0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.445225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 900/1816] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:36 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:36.449231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca024a58-1ba5-40b6-afee-2f563ac3eef3 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.460551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca024a58-1ba5-40b6-afee-2f563ac3eef3 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.484109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:36.517184 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd81d0c-f939-45cc-a447-c73a4d51c6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.517848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd81d0c-f939-45cc-a447-c73a4d51c6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.518134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd81d0c-f939-45cc-a447-c73a4d51c6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.518412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd81d0c-f939-45cc-a447-c73a4d51c6f6 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.519140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd81d0c-f939-45cc-a447-c73a4d51c6f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.521042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd81d0c-f939-45cc-a447-c73a4d51c6f6 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.540217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 917/1817] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:36 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:36.544226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6ffa7-855a-4e33-88eb-4749f09035f0 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.554400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6ffa7-855a-4e33-88eb-4749f09035f0 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.559029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4b4fc-1359-4f20-bcd4-4fbbdc436b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.559685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4b4fc-1359-4f20-bcd4-4fbbdc436b3f admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.559969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4b4fc-1359-4f20-bcd4-4fbbdc436b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.560248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4b4fc-1359-4f20-bcd4-4fbbdc436b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.560972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4b4fc-1359-4f20-bcd4-4fbbdc436b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.563119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4b4fc-1359-4f20-bcd4-4fbbdc436b3f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.573607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 901/1818] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:36.577586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd81d0c-f939-45cc-a447-c73a4d51c6f6 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.587987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd81d0c-f939-45cc-a447-c73a4d51c6f6 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.592622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351f819-b9fb-4614-9956-b7fad2c51571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.593499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351f819-b9fb-4614-9956-b7fad2c51571 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.593779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351f819-b9fb-4614-9956-b7fad2c51571 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.594052 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351f819-b9fb-4614-9956-b7fad2c51571 admin admin] PATH_INFO: `/v3/projects/6fcc94dd8324440bb4f86524b3339372/users/a964b9c7047944519ed6a65673bd34d1/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.608487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351f819-b9fb-4614-9956-b7fad2c51571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fcc94dd8324440bb4f86524b3339372, user_id=a964b9c7047944519ed6a65673bd34d1, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.611152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351f819-b9fb-4614-9956-b7fad2c51571 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.625329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 918/1819] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:36 2022] PUT /identity/v3/projects/6fcc94dd8324440bb4f86524b3339372/users/a964b9c7047944519ed6a65673bd34d1/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:36.629145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4b4fc-1359-4f20-bcd4-4fbbdc436b3f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.639085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4b4fc-1359-4f20-bcd4-4fbbdc436b3f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.696549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63a612-af84-4c2e-88f2-9b6cada33320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.697216 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63a612-af84-4c2e-88f2-9b6cada33320 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.697499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63a612-af84-4c2e-88f2-9b6cada33320 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.697776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63a612-af84-4c2e-88f2-9b6cada33320 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.698494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63a612-af84-4c2e-88f2-9b6cada33320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.700124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63a612-af84-4c2e-88f2-9b6cada33320 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.710957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 902/1820] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:36.715181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351f819-b9fb-4614-9956-b7fad2c51571 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.725943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351f819-b9fb-4614-9956-b7fad2c51571 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.731131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c350e3d-7373-4f18-a8f3-1e6640188637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.732097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c350e3d-7373-4f18-a8f3-1e6640188637 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.732445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c350e3d-7373-4f18-a8f3-1e6640188637 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.732788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c350e3d-7373-4f18-a8f3-1e6640188637 admin admin] PATH_INFO: `/v3/projects/6fcc94dd8324440bb4f86524b3339372/users/a964b9c7047944519ed6a65673bd34d1/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.735381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c350e3d-7373-4f18-a8f3-1e6640188637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fcc94dd8324440bb4f86524b3339372, user_id=a964b9c7047944519ed6a65673bd34d1, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.738095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c350e3d-7373-4f18-a8f3-1e6640188637 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.750668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 919/1821] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:36 2022] PUT /identity/v3/projects/6fcc94dd8324440bb4f86524b3339372/users/a964b9c7047944519ed6a65673bd34d1/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:36.761746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96cb16-d8b3-4194-a0f8-880675fdd13f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.762035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96cb16-d8b3-4194-a0f8-880675fdd13f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.762314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96cb16-d8b3-4194-a0f8-880675fdd13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.788312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8e96cb16-d8b3-4194-a0f8-880675fdd13f None None] MFA Rules not processed for user `a964b9c7047944519ed6a65673bd34d1`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:36.789605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8e96cb16-d8b3-4194-a0f8-880675fdd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:36.875941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 903/1822] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:36.885334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c350e3d-7373-4f18-a8f3-1e6640188637 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:36.887065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2c350e3d-7373-4f18-a8f3-1e6640188637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:36.948196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c350e3d-7373-4f18-a8f3-1e6640188637 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:36.951876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f0169-d590-4811-b98b-dcc1acb8ce15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:36.952613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f0169-d590-4811-b98b-dcc1acb8ce15 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:36.952901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f0169-d590-4811-b98b-dcc1acb8ce15 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:36.953186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f0169-d590-4811-b98b-dcc1acb8ce15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:36.962400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f0169-d590-4811-b98b-dcc1acb8ce15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:36.964061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f0169-d590-4811-b98b-dcc1acb8ce15 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:36.973958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 920/1823] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:37.519817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96cb16-d8b3-4194-a0f8-880675fdd13f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:37.529429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:37.582103 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96cb16-d8b3-4194-a0f8-880675fdd13f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:37.586500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:37.587393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:37.587676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:37.587950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] PATH_INFO: `/v3/users/7210b91378e84e5887d867a0502104eb` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:37.589114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7210b91378e84e5887d867a0502104eb)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:37.591115 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:37.636578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7210b91378e84e5887d867a0502104eb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:37.644152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7210b91378e84e5887d867a0502104eb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:37.651177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7210b91378e84e5887d867a0502104eb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:37.651798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7210b91378e84e5887d867a0502104eb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:37.652397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7210b91378e84e5887d867a0502104eb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:37.669657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 904/1824] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:37 2022] DELETE /identity/v3/users/7210b91378e84e5887d867a0502104eb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:37.674146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475bec3-ffaa-477a-a30f-705109ecc25d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:37.674566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475bec3-ffaa-477a-a30f-705109ecc25d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:37.674703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475bec3-ffaa-477a-a30f-705109ecc25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:37.691890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-a475bec3-ffaa-477a-a30f-705109ecc25d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:37.693082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a475bec3-ffaa-477a-a30f-705109ecc25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:37.751907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 921/1825] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:37.761483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:37.762926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:37.824063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16831ca4-8a83-4bbb-9955-30186c028a8f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:37.828217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae94e-a70c-4675-b982-7516bd340d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:37.828863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae94e-a70c-4675-b982-7516bd340d42 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:37.829152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae94e-a70c-4675-b982-7516bd340d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:37.829424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae94e-a70c-4675-b982-7516bd340d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:37.838863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ae94e-a70c-4675-b982-7516bd340d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:37.840297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ae94e-a70c-4675-b982-7516bd340d42 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:37.850706 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 905/1826] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:38.326473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475bec3-ffaa-477a-a30f-705109ecc25d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:38.328296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a475bec3-ffaa-477a-a30f-705109ecc25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:38.341251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475bec3-ffaa-477a-a30f-705109ecc25d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:38.345932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:38.346889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:38.347120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:38.347419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] PATH_INFO: `/v3/projects/c09be2db83974b65bc31b96f31e7587b` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:38.348722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09be2db83974b65bc31b96f31e7587b)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:38.350669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:38.367999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:38.397911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09be2db83974b65bc31b96f31e7587b', 'request_id': 'req-17ed294b-f214-43b0-bee5-50466a6340f7'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.404027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09be2db83974b65bc31b96f31e7587b', 'request_id': 'req-17ed294b-f214-43b0-bee5-50466a6340f7'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.411070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09be2db83974b65bc31b96f31e7587b', 'request_id': 'req-17ed294b-f214-43b0-bee5-50466a6340f7'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.423424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] The token cache is being invalidate because project c09be2db83974b65bc31b96f31e7587b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:38.423715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.424359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.426075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 922/1827] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:38 2022] DELETE /identity/v3/projects/c09be2db83974b65bc31b96f31e7587b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:38.429834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae94e-a70c-4675-b982-7516bd340d42 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:38.440417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae94e-a70c-4675-b982-7516bd340d42 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:38.474551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:38.507039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:38.508127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:38.508413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:38.508691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] PATH_INFO: `/v3/users/a964b9c7047944519ed6a65673bd34d1` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:38.509884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a964b9c7047944519ed6a65673bd34d1)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:38.511833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:38.559743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a964b9c7047944519ed6a65673bd34d1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.569637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a964b9c7047944519ed6a65673bd34d1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.577042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a964b9c7047944519ed6a65673bd34d1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.577752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a964b9c7047944519ed6a65673bd34d1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.578423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-befd5c18-36e9-4c77-99ae-aff835188c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a964b9c7047944519ed6a65673bd34d1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:38.596583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 906/1828] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:38 2022] DELETE /identity/v3/users/a964b9c7047944519ed6a65673bd34d1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:38.859416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:38.860861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:38.875983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed294b-f214-43b0-bee5-50466a6340f7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:38.933562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:38.934420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:38.934787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:38.935076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] PATH_INFO: `/v3/projects/6fcc94dd8324440bb4f86524b3339372` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:38.936231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fcc94dd8324440bb4f86524b3339372)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:38.938662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:38.993181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fcc94dd8324440bb4f86524b3339372', 'request_id': 'req-df05508f-365c-4548-ad54-5d4a3152acd9'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:39.000914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fcc94dd8324440bb4f86524b3339372', 'request_id': 'req-df05508f-365c-4548-ad54-5d4a3152acd9'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:39.009212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fcc94dd8324440bb4f86524b3339372', 'request_id': 'req-df05508f-365c-4548-ad54-5d4a3152acd9'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:39.026068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] The token cache is being invalidate because project 6fcc94dd8324440bb4f86524b3339372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:39.026404 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:39.027208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-df05508f-365c-4548-ad54-5d4a3152acd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:39.029332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 923/1829] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:38 2022] DELETE /identity/v3/projects/6fcc94dd8324440bb4f86524b3339372 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:39.034442 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc50c95-bb3d-428c-a918-f741822236b6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.035092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc50c95-bb3d-428c-a918-f741822236b6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.035186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc50c95-bb3d-428c-a918-f741822236b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.058423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-1dc50c95-bb3d-428c-a918-f741822236b6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:39.059778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1dc50c95-bb3d-428c-a918-f741822236b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:39.129613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 907/1830] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:39.137542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c6ab9-ed60-41ed-8913-34f79cdea108 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.137836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c6ab9-ed60-41ed-8913-34f79cdea108 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.138122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c6ab9-ed60-41ed-8913-34f79cdea108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.160816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e46c6ab9-ed60-41ed-8913-34f79cdea108 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:39.162606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e46c6ab9-ed60-41ed-8913-34f79cdea108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:39.170830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 924/1831] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:39.175586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc50c95-bb3d-428c-a918-f741822236b6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:39.187422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc50c95-bb3d-428c-a918-f741822236b6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:39.192280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f804fc-ec01-419f-9202-64fe981e6ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:39.192945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f804fc-ec01-419f-9202-64fe981e6ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.193236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f804fc-ec01-419f-9202-64fe981e6ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.193508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f804fc-ec01-419f-9202-64fe981e6ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.194232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f804fc-ec01-419f-9202-64fe981e6ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:50:39.195131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f804fc-ec01-419f-9202-64fe981e6ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:39.196836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f804fc-ec01-419f-9202-64fe981e6ba5 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:39.213434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 908/1832] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:50:39.214864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c6ab9-ed60-41ed-8913-34f79cdea108 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:39.232583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c6ab9-ed60-41ed-8913-34f79cdea108 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:39.238844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcd2ef-4598-4f27-8f38-87a11a52637f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:39.239513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcd2ef-4598-4f27-8f38-87a11a52637f admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.239795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcd2ef-4598-4f27-8f38-87a11a52637f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.240077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcd2ef-4598-4f27-8f38-87a11a52637f admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.240763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcd2ef-4598-4f27-8f38-87a11a52637f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:39.242661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcd2ef-4598-4f27-8f38-87a11a52637f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:39.272531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 925/1833] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:39 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:39.273892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f804fc-ec01-419f-9202-64fe981e6ba5 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:39.285050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f804fc-ec01-419f-9202-64fe981e6ba5 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:39.323062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:39.353111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd2d2b-e126-4111-a8bf-c9546c5ffcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:39.354031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2d2b-e126-4111-a8bf-c9546c5ffcdd admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.354386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2d2b-e126-4111-a8bf-c9546c5ffcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.354878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2d2b-e126-4111-a8bf-c9546c5ffcdd admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.355590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd2d2b-e126-4111-a8bf-c9546c5ffcdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:39.357983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd2d2b-e126-4111-a8bf-c9546c5ffcdd admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:39.380679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 909/1834] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:39 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:50:39.389491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcd2ef-4598-4f27-8f38-87a11a52637f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:39.395419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcd2ef-4598-4f27-8f38-87a11a52637f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:39.400265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34258b0b-9676-4d90-8a9e-a65da490b497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:39.400999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258b0b-9676-4d90-8a9e-a65da490b497 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.401417 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258b0b-9676-4d90-8a9e-a65da490b497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.401765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258b0b-9676-4d90-8a9e-a65da490b497 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.402638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34258b0b-9676-4d90-8a9e-a65da490b497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:39.404248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34258b0b-9676-4d90-8a9e-a65da490b497 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:39.406181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:39.420952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd2d2b-e126-4111-a8bf-c9546c5ffcdd admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:39.422767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 926/1835] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:39.431789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd2d2b-e126-4111-a8bf-c9546c5ffcdd admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:39.436798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8125b-b542-4810-ba31-62f27976cb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:39.439497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8125b-b542-4810-ba31-62f27976cb75 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.439959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8125b-b542-4810-ba31-62f27976cb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.440309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8125b-b542-4810-ba31-62f27976cb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.441159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8125b-b542-4810-ba31-62f27976cb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:39.442927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8125b-b542-4810-ba31-62f27976cb75 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:39.454761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 910/1836] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:39.458082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34258b0b-9676-4d90-8a9e-a65da490b497 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:39.470474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34258b0b-9676-4d90-8a9e-a65da490b497 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:39.482023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265b4f2-bd0d-4e01-b293-2c2421e21517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:39.483151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265b4f2-bd0d-4e01-b293-2c2421e21517 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.484501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265b4f2-bd0d-4e01-b293-2c2421e21517 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.484501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265b4f2-bd0d-4e01-b293-2c2421e21517 admin admin] PATH_INFO: `/v3/projects/b3e7025f89b941c18111abaedd708554/users/fbfba31fa3834b559828f527ab7f1a9a/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.501477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265b4f2-bd0d-4e01-b293-2c2421e21517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e7025f89b941c18111abaedd708554, user_id=fbfba31fa3834b559828f527ab7f1a9a, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:39.504160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265b4f2-bd0d-4e01-b293-2c2421e21517 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:39.519582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 927/1837] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:39 2022] PUT /identity/v3/projects/b3e7025f89b941c18111abaedd708554/users/fbfba31fa3834b559828f527ab7f1a9a/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:39.530314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c907fde-da9f-4f2c-bade-ab4b8de7f1fc None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.530689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c907fde-da9f-4f2c-bade-ab4b8de7f1fc None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.531042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c907fde-da9f-4f2c-bade-ab4b8de7f1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.559341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9c907fde-da9f-4f2c-bade-ab4b8de7f1fc None None] MFA Rules not processed for user `fbfba31fa3834b559828f527ab7f1a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:39.560578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9c907fde-da9f-4f2c-bade-ab4b8de7f1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:39.631577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 911/1838] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:39.642708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265b4f2-bd0d-4e01-b293-2c2421e21517 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:39.645078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3265b4f2-bd0d-4e01-b293-2c2421e21517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:39.708074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265b4f2-bd0d-4e01-b293-2c2421e21517 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:39.711790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720a2fd-2bf0-40ba-82c9-43ab12ce000e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:39.712538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720a2fd-2bf0-40ba-82c9-43ab12ce000e service glance] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:39.712891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720a2fd-2bf0-40ba-82c9-43ab12ce000e service glance] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:39.713280 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720a2fd-2bf0-40ba-82c9-43ab12ce000e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:39.722324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720a2fd-2bf0-40ba-82c9-43ab12ce000e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:39.723909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720a2fd-2bf0-40ba-82c9-43ab12ce000e service glance] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:39.737993 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 928/1839] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:40.060331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14322eac-0625-4b15-ba5a-d8608b7e6b66 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:40.061193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14322eac-0625-4b15-ba5a-d8608b7e6b66 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:40.061484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14322eac-0625-4b15-ba5a-d8608b7e6b66 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:40.063617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 912/1840] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:50:40.077608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f433b-10aa-47c2-8e2b-acb9063d0e4c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:40.078197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f433b-10aa-47c2-8e2b-acb9063d0e4c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:40.078302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f433b-10aa-47c2-8e2b-acb9063d0e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:40.097358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4d3f433b-10aa-47c2-8e2b-acb9063d0e4c None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:40.098724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4d3f433b-10aa-47c2-8e2b-acb9063d0e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:40.185077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 929/1841] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:50:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:40.199708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14322eac-0625-4b15-ba5a-d8608b7e6b66 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:40.201591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-14322eac-0625-4b15-ba5a-d8608b7e6b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:40.223967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14322eac-0625-4b15-ba5a-d8608b7e6b66 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:40.241218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c025b5-86d8-4f3d-8e13-64b2807630b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:40.242126 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c025b5-86d8-4f3d-8e13-64b2807630b1 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:40.242761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c025b5-86d8-4f3d-8e13-64b2807630b1 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:40.243253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c025b5-86d8-4f3d-8e13-64b2807630b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:40.266778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c025b5-86d8-4f3d-8e13-64b2807630b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:40.271360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c025b5-86d8-4f3d-8e13-64b2807630b1 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:40.288566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 913/1842] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:41.515261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f433b-10aa-47c2-8e2b-acb9063d0e4c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:41.517968 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4d3f433b-10aa-47c2-8e2b-acb9063d0e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:41.611029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f433b-10aa-47c2-8e2b-acb9063d0e4c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:41.619586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:41.620754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:41.621036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:41.621383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] PATH_INFO: `/v3/users/11437e5915ae466687a2ef6f8ae551bc` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:41.622866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11437e5915ae466687a2ef6f8ae551bc)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:41.625244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:41.633367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:41.688809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11437e5915ae466687a2ef6f8ae551bc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.703139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11437e5915ae466687a2ef6f8ae551bc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.714377 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11437e5915ae466687a2ef6f8ae551bc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.715148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11437e5915ae466687a2ef6f8ae551bc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.745405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92f847ef-5f61-42d1-8952-7e9a94df63a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11437e5915ae466687a2ef6f8ae551bc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.755746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 930/1843] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:41 2022] DELETE /identity/v3/users/11437e5915ae466687a2ef6f8ae551bc => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:41.757918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b79cdd-fa0a-4b3e-b1b0-a3367f7fea04 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:41.758033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c025b5-86d8-4f3d-8e13-64b2807630b1 service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:41.758493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b79cdd-fa0a-4b3e-b1b0-a3367f7fea04 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:41.758493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b79cdd-fa0a-4b3e-b1b0-a3367f7fea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:41.760086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-31c025b5-86d8-4f3d-8e13-64b2807630b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:41.777095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c025b5-86d8-4f3d-8e13-64b2807630b1 service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:41.778838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-80b79cdd-fa0a-4b3e-b1b0-a3367f7fea04 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:41.780305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-80b79cdd-fa0a-4b3e-b1b0-a3367f7fea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:41.869927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:41.871005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:41.871139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:41.871405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] PATH_INFO: `/v3/users/a9d24334a00a4df09326e20ea6dc1fa3` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:41.872864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d24334a00a4df09326e20ea6dc1fa3)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:41.874864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:41.893331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 931/1844] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:41.904948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b79cdd-fa0a-4b3e-b1b0-a3367f7fea04 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:41.907289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-80b79cdd-fa0a-4b3e-b1b0-a3367f7fea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:41.930427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:41.945279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d24334a00a4df09326e20ea6dc1fa3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.955493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d24334a00a4df09326e20ea6dc1fa3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.964137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d24334a00a4df09326e20ea6dc1fa3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.966135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d24334a00a4df09326e20ea6dc1fa3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.966780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-085968c0-51c2-4b80-8cf2-fa9837ca04e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d24334a00a4df09326e20ea6dc1fa3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:41.977700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b79cdd-fa0a-4b3e-b1b0-a3367f7fea04 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:41.983355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:41.984370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:41.984825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:41.985180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:41.987203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 914/1845] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:41 2022] DELETE /identity/v3/users/a9d24334a00a4df09326e20ea6dc1fa3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:41.987327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:41.991534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebd460-3585-4b16-8a70-94bf1bd3bb89 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:41.991832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebd460-3585-4b16-8a70-94bf1bd3bb89 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:41.992119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebd460-3585-4b16-8a70-94bf1bd3bb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:42.005898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:42.007624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:42.011825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-49ebd460-3585-4b16-8a70-94bf1bd3bb89 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:42.013069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-49ebd460-3585-4b16-8a70-94bf1bd3bb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:42.021024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 932/1846] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:50:42.090899 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 915/1847] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:42.104434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:42.105980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:42.119820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21728334-ed82-4a93-a6f4-91e84359eeb2 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:42.157945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:42.197866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3df874-1476-4c65-9ba2-32409326aec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:42.198538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df874-1476-4c65-9ba2-32409326aec3 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:42.198861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df874-1476-4c65-9ba2-32409326aec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:42.199222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df874-1476-4c65-9ba2-32409326aec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:42.212746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3df874-1476-4c65-9ba2-32409326aec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:42.214259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3df874-1476-4c65-9ba2-32409326aec3 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:42.228698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 933/1848] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:42.552325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebd460-3585-4b16-8a70-94bf1bd3bb89 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:42.554300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-49ebd460-3585-4b16-8a70-94bf1bd3bb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:42.570271 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebd460-3585-4b16-8a70-94bf1bd3bb89 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:42.584323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:42.587918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:42.588286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:42.588580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] PATH_INFO: `/v3/projects/a5425fc09f574ff7a4f8a2c89d8fcd9b` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:42.590409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5425fc09f574ff7a4f8a2c89d8fcd9b)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:42.593370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:42.608065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3df874-1476-4c65-9ba2-32409326aec3 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:42.610717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ad3df874-1476-4c65-9ba2-32409326aec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:42.627540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3df874-1476-4c65-9ba2-32409326aec3 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:42.646208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:42.647833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:42.650975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:42.651332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] PATH_INFO: `/v3/projects/3e6cd256adae4dec948b931f9e773f32` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:42.653045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e6cd256adae4dec948b931f9e773f32)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:42.655235 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:42.666649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5425fc09f574ff7a4f8a2c89d8fcd9b', 'request_id': 'req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.674207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5425fc09f574ff7a4f8a2c89d8fcd9b', 'request_id': 'req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.683589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5425fc09f574ff7a4f8a2c89d8fcd9b', 'request_id': 'req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.699034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] The token cache is being invalidate because project a5425fc09f574ff7a4f8a2c89d8fcd9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:42.699334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.700520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.701882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 916/1849] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:42 2022] DELETE /identity/v3/projects/a5425fc09f574ff7a4f8a2c89d8fcd9b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:42.706397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:42.707857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:42.724239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e6cd256adae4dec948b931f9e773f32', 'request_id': 'req-8f332fe6-b51e-4950-afab-710c3dac8cfe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.731487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e6cd256adae4dec948b931f9e773f32', 'request_id': 'req-8f332fe6-b51e-4950-afab-710c3dac8cfe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.741843 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e6cd256adae4dec948b931f9e773f32', 'request_id': 'req-8f332fe6-b51e-4950-afab-710c3dac8cfe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.758817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] The token cache is being invalidate because project 3e6cd256adae4dec948b931f9e773f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:42.759219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.760127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:42.761957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 934/1850] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:42 2022] DELETE /identity/v3/projects/3e6cd256adae4dec948b931f9e773f32 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:42.763675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:42.765892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:42.767303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:42.798789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07cfd2-7d5c-4cdb-a747-a044acbe45b6 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:42.900056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f332fe6-b51e-4950-afab-710c3dac8cfe admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:42.923466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:42.926041 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:42.926340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:42.927389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] PATH_INFO: `/v3/users/f280008a3a7c4002acedb29627af4886` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:42.930074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f280008a3a7c4002acedb29627af4886)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:42.931687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:42.933868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:42.934269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:42.934481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:42.935155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] PATH_INFO: `/v3/users/e74cd68a5ff942559a771da8159a75ea` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:42.936756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74cd68a5ff942559a771da8159a75ea)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:42.941015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:43.062415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f280008a3a7c4002acedb29627af4886'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.072867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74cd68a5ff942559a771da8159a75ea'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.077894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f280008a3a7c4002acedb29627af4886'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.083525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:43.090056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74cd68a5ff942559a771da8159a75ea'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.100103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f280008a3a7c4002acedb29627af4886'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.101701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f280008a3a7c4002acedb29627af4886'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.114502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74cd68a5ff942559a771da8159a75ea'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.115796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74cd68a5ff942559a771da8159a75ea'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.116980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74cd68a5ff942559a771da8159a75ea'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.130980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f280008a3a7c4002acedb29627af4886'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:43.147491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 935/1851] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:42 2022] DELETE /identity/v3/users/f280008a3a7c4002acedb29627af4886 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:43.166450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 917/1852] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:50:42 2022] DELETE /identity/v3/users/e74cd68a5ff942559a771da8159a75ea => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:43.999949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.002160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:44.007726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.009429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:44.081980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1977bf-340e-4f3c-a3c8-a2594d1a55c1 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.098919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5de807-4b04-4aa7-b2d3-dd9b0047049f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.105165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.106513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.107006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.107424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] PATH_INFO: `/v3/projects/09462e301a6d4a9db49be1762803c407` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.109964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09462e301a6d4a9db49be1762803c407)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.112776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.116142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:44.182187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09462e301a6d4a9db49be1762803c407', 'request_id': 'req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.188718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.189789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.190639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.190973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] PATH_INFO: `/v3/projects/3e449c795d1a46caa00f0dd7b4cc66af` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.192765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e449c795d1a46caa00f0dd7b4cc66af)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.192972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09462e301a6d4a9db49be1762803c407', 'request_id': 'req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.195547 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.207394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09462e301a6d4a9db49be1762803c407', 'request_id': 'req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.229700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] The token cache is being invalidate because project 09462e301a6d4a9db49be1762803c407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:44.230385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.230925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-119d058a-4cd9-4a6d-a9ee-94da4b436dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.233378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 918/1853] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:44 2022] DELETE /identity/v3/projects/09462e301a6d4a9db49be1762803c407 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:44.239670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72f4c9-a13d-4bbc-81f3-fbb1b294d40d None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.240028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72f4c9-a13d-4bbc-81f3-fbb1b294d40d None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.241140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72f4c9-a13d-4bbc-81f3-fbb1b294d40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.266994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e449c795d1a46caa00f0dd7b4cc66af', 'request_id': 'req-074e81e1-c914-4737-a4ec-45a2c00dc742'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.273754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ea72f4c9-a13d-4bbc-81f3-fbb1b294d40d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:44.275765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e449c795d1a46caa00f0dd7b4cc66af', 'request_id': 'req-074e81e1-c914-4737-a4ec-45a2c00dc742'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.276119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ea72f4c9-a13d-4bbc-81f3-fbb1b294d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:44.290460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e449c795d1a46caa00f0dd7b4cc66af', 'request_id': 'req-074e81e1-c914-4737-a4ec-45a2c00dc742'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.308093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] The token cache is being invalidate because project 3e449c795d1a46caa00f0dd7b4cc66af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:50:44.308385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.309072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-074e81e1-c914-4737-a4ec-45a2c00dc742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:50:44.310767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 936/1854] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:50:43 2022] DELETE /identity/v3/projects/3e449c795d1a46caa00f0dd7b4cc66af => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:44.319168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd66177-49f5-416d-a0cf-782a739d4162 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.319603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd66177-49f5-416d-a0cf-782a739d4162 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.319895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd66177-49f5-416d-a0cf-782a739d4162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.344916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2fd66177-49f5-416d-a0cf-782a739d4162 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:44.347887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2fd66177-49f5-416d-a0cf-782a739d4162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:44.352037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:44.360905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 919/1855] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:44.368694 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132308d2-e502-4610-a330-459ea64a0522 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.368988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132308d2-e502-4610-a330-459ea64a0522 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.369382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132308d2-e502-4610-a330-459ea64a0522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.387692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-132308d2-e502-4610-a330-459ea64a0522 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:44.388915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-132308d2-e502-4610-a330-459ea64a0522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:44.398904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 920/1856] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:44.403819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132308d2-e502-4610-a330-459ea64a0522 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.414622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132308d2-e502-4610-a330-459ea64a0522 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.419439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7f2e7-c846-49d8-a62c-f1f1475cef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.420106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7f2e7-c846-49d8-a62c-f1f1475cef83 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.420394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7f2e7-c846-49d8-a62c-f1f1475cef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.420674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7f2e7-c846-49d8-a62c-f1f1475cef83 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.422255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7f2e7-c846-49d8-a62c-f1f1475cef83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:50:44.422600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7f2e7-c846-49d8-a62c-f1f1475cef83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.423986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 937/1857] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:44.426319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7f2e7-c846-49d8-a62c-f1f1475cef83 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.441452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba777f6-3749-43fd-94c4-340899393821 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.441820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba777f6-3749-43fd-94c4-340899393821 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.442173 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba777f6-3749-43fd-94c4-340899393821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.443383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 921/1858] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:50:44.447641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7f2e7-c846-49d8-a62c-f1f1475cef83 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.460581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7f2e7-c846-49d8-a62c-f1f1475cef83 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.467264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9ba777f6-3749-43fd-94c4-340899393821 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:44.469000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53b5c8-d07e-4829-97f2-9ad795d0a6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.469385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9ba777f6-3749-43fd-94c4-340899393821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:44.469683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53b5c8-d07e-4829-97f2-9ad795d0a6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.469970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53b5c8-d07e-4829-97f2-9ad795d0a6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.470257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53b5c8-d07e-4829-97f2-9ad795d0a6d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.471056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53b5c8-d07e-4829-97f2-9ad795d0a6d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.473550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53b5c8-d07e-4829-97f2-9ad795d0a6d9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.491084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 938/1859] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:44.496663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba777f6-3749-43fd-94c4-340899393821 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.510598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba777f6-3749-43fd-94c4-340899393821 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.517976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc82fd-8c98-4a87-851b-8842f697c1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.518814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc82fd-8c98-4a87-851b-8842f697c1ca admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.519376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc82fd-8c98-4a87-851b-8842f697c1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.520052 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc82fd-8c98-4a87-851b-8842f697c1ca admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.521219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc82fd-8c98-4a87-851b-8842f697c1ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:50:44.521924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc82fd-8c98-4a87-851b-8842f697c1ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.524106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc82fd-8c98-4a87-851b-8842f697c1ca admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.531633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 922/1860] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:44 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:50:44.535968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53b5c8-d07e-4829-97f2-9ad795d0a6d9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.544369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 939/1861] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:50:44.549011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc82fd-8c98-4a87-851b-8842f697c1ca admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.573550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc82fd-8c98-4a87-851b-8842f697c1ca admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.643996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:44.659224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10987181-92eb-4231-adb3-425dd29ac8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.659956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10987181-92eb-4231-adb3-425dd29ac8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.660241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10987181-92eb-4231-adb3-425dd29ac8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.660528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10987181-92eb-4231-adb3-425dd29ac8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.661223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10987181-92eb-4231-adb3-425dd29ac8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.663777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10987181-92eb-4231-adb3-425dd29ac8e3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.664330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53b5c8-d07e-4829-97f2-9ad795d0a6d9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.671488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef3c2f-a172-4274-94ce-ec679f0a93f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.672191 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef3c2f-a172-4274-94ce-ec679f0a93f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.672915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef3c2f-a172-4274-94ce-ec679f0a93f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.673204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef3c2f-a172-4274-94ce-ec679f0a93f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.674404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef3c2f-a172-4274-94ce-ec679f0a93f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.676164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef3c2f-a172-4274-94ce-ec679f0a93f0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.700428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 940/1862] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:44 2022] POST /identity/v3/projects => generated 409 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:44.705147 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10987181-92eb-4231-adb3-425dd29ac8e3 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.711884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 923/1863] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:44 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:44.716430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef3c2f-a172-4274-94ce-ec679f0a93f0 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.729211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef3c2f-a172-4274-94ce-ec679f0a93f0 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.794937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10987181-92eb-4231-adb3-425dd29ac8e3 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.802625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9085-3789-42a6-8fc9-ae7a65dba390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.803375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9085-3789-42a6-8fc9-ae7a65dba390 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.803660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9085-3789-42a6-8fc9-ae7a65dba390 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.803936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9085-3789-42a6-8fc9-ae7a65dba390 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.804643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9085-3789-42a6-8fc9-ae7a65dba390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.806591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9085-3789-42a6-8fc9-ae7a65dba390 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.807474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd97621-b446-4bc5-8e72-75b3be979746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.808156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd97621-b446-4bc5-8e72-75b3be979746 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.808442 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd97621-b446-4bc5-8e72-75b3be979746 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.808725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd97621-b446-4bc5-8e72-75b3be979746 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.809455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd97621-b446-4bc5-8e72-75b3be979746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.811099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd97621-b446-4bc5-8e72-75b3be979746 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.823548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 924/1864] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:44.831263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd97621-b446-4bc5-8e72-75b3be979746 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.836355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 941/1865] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:44 2022] POST /identity/v3/users => generated 305 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:44.839178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9085-3789-42a6-8fc9-ae7a65dba390 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.842936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd97621-b446-4bc5-8e72-75b3be979746 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.848825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab343f-9133-4ca2-ad5f-4cbe2a0d1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.849191 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab343f-9133-4ca2-ad5f-4cbe2a0d1fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.849480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab343f-9133-4ca2-ad5f-4cbe2a0d1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.849762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab343f-9133-4ca2-ad5f-4cbe2a0d1fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.850658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab343f-9133-4ca2-ad5f-4cbe2a0d1fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.852322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab343f-9133-4ca2-ad5f-4cbe2a0d1fe2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.859713 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9085-3789-42a6-8fc9-ae7a65dba390 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.865683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc585d-a626-4f34-9f03-87d0768ec1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.866297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc585d-a626-4f34-9f03-87d0768ec1ad admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.866582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc585d-a626-4f34-9f03-87d0768ec1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.866900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc585d-a626-4f34-9f03-87d0768ec1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.867811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 925/1866] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:44.870842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc585d-a626-4f34-9f03-87d0768ec1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.873558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab343f-9133-4ca2-ad5f-4cbe2a0d1fe2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.877467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc585d-a626-4f34-9f03-87d0768ec1ad admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.879435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:44.897614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab343f-9133-4ca2-ad5f-4cbe2a0d1fe2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.899546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 942/1867] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:44.901946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc585d-a626-4f34-9f03-87d0768ec1ad admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.908573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f4641-7247-4004-b55a-8a39c2914a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.909664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4641-7247-4004-b55a-8a39c2914a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.910019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4641-7247-4004-b55a-8a39c2914a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.910375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4641-7247-4004-b55a-8a39c2914a4a admin admin] PATH_INFO: `/v3/projects/1d9de3185e48434ea92808b08bae27ab/users/651249ee12184319b66f74d3652648f6/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.920026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc585d-a626-4f34-9f03-87d0768ec1ad admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.925861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce3f97-e131-431c-ab56-722d9ca8adc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:44.927057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3f97-e131-431c-ab56-722d9ca8adc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.927526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3f97-e131-431c-ab56-722d9ca8adc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.927970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3f97-e131-431c-ab56-722d9ca8adc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:44.929053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce3f97-e131-431c-ab56-722d9ca8adc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.931337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce3f97-e131-431c-ab56-722d9ca8adc7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.936303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f4641-7247-4004-b55a-8a39c2914a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9de3185e48434ea92808b08bae27ab, user_id=651249ee12184319b66f74d3652648f6, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:44.939460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f4641-7247-4004-b55a-8a39c2914a4a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:44.949557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 943/1868] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:44.954634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce3f97-e131-431c-ab56-722d9ca8adc7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:44.962778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 926/1869] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:44 2022] PUT /identity/v3/projects/1d9de3185e48434ea92808b08bae27ab/users/651249ee12184319b66f74d3652648f6/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:44.975406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d509a3-80d1-473d-8d3e-28b8ce04336c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:44.975596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d509a3-80d1-473d-8d3e-28b8ce04336c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:44.975974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce3f97-e131-431c-ab56-722d9ca8adc7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:44.976752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d509a3-80d1-473d-8d3e-28b8ce04336c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.015397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-62d509a3-80d1-473d-8d3e-28b8ce04336c None None] MFA Rules not processed for user `651249ee12184319b66f74d3652648f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:45.016630 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-62d509a3-80d1-473d-8d3e-28b8ce04336c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:45.055136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084f6a0-0307-4262-8d33-ba0695909c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.056279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084f6a0-0307-4262-8d33-ba0695909c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.056662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084f6a0-0307-4262-8d33-ba0695909c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.057342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084f6a0-0307-4262-8d33-ba0695909c17 admin admin] PATH_INFO: `/v3/projects/abad183a4e564844952333f4cbecea03/users/9ba081cb43674c26947b05412d9184f8/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.081668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084f6a0-0307-4262-8d33-ba0695909c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abad183a4e564844952333f4cbecea03, user_id=9ba081cb43674c26947b05412d9184f8, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.084550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084f6a0-0307-4262-8d33-ba0695909c17 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.098448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 927/1870] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:45.103652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d509a3-80d1-473d-8d3e-28b8ce04336c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.110825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 944/1871] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:44 2022] PUT /identity/v3/projects/abad183a4e564844952333f4cbecea03/users/9ba081cb43674c26947b05412d9184f8/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:45.116859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d509a3-80d1-473d-8d3e-28b8ce04336c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.118059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bace2a0-9e3e-4fe7-9d12-4e9260a09170 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.119064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bace2a0-9e3e-4fe7-9d12-4e9260a09170 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.119533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bace2a0-9e3e-4fe7-9d12-4e9260a09170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.160423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6bace2a0-9e3e-4fe7-9d12-4e9260a09170 None None] MFA Rules not processed for user `9ba081cb43674c26947b05412d9184f8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:45.160629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:45.162462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6bace2a0-9e3e-4fe7-9d12-4e9260a09170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:45.201057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0471e50-1e23-402e-9e27-13289d36bcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.201935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0471e50-1e23-402e-9e27-13289d36bcbf admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.202402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0471e50-1e23-402e-9e27-13289d36bcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.202922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0471e50-1e23-402e-9e27-13289d36bcbf admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.203759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0471e50-1e23-402e-9e27-13289d36bcbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.205991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0471e50-1e23-402e-9e27-13289d36bcbf admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.242799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 928/1872] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:45 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:45.244042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0471e50-1e23-402e-9e27-13289d36bcbf admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.262707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 945/1873] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:45.267540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bace2a0-9e3e-4fe7-9d12-4e9260a09170 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.282948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bace2a0-9e3e-4fe7-9d12-4e9260a09170 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.316651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0471e50-1e23-402e-9e27-13289d36bcbf admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.321717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79099042-932b-4207-a8da-18cd82ec6fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.322655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79099042-932b-4207-a8da-18cd82ec6fee admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.323107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79099042-932b-4207-a8da-18cd82ec6fee admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.323587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79099042-932b-4207-a8da-18cd82ec6fee admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.324348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79099042-932b-4207-a8da-18cd82ec6fee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.326426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79099042-932b-4207-a8da-18cd82ec6fee admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.339496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:45.347959 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcf248-042c-4537-abad-25966e215814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.348878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcf248-042c-4537-abad-25966e215814 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.349344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcf248-042c-4537-abad-25966e215814 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.349864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcf248-042c-4537-abad-25966e215814 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.350563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcf248-042c-4537-abad-25966e215814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.350909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 929/1874] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:45 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:50:45.352734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcf248-042c-4537-abad-25966e215814 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.354684 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79099042-932b-4207-a8da-18cd82ec6fee admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.365415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79099042-932b-4207-a8da-18cd82ec6fee admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.370465 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc1047-7c07-4c93-93b8-0b699e8a1799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.371825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc1047-7c07-4c93-93b8-0b699e8a1799 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.372131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc1047-7c07-4c93-93b8-0b699e8a1799 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.372396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc1047-7c07-4c93-93b8-0b699e8a1799 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.373291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc1047-7c07-4c93-93b8-0b699e8a1799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.375152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc1047-7c07-4c93-93b8-0b699e8a1799 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.386440 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 946/1875] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:50:45 2022] POST /identity/v3/projects => generated 407 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:45.386640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 930/1876] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:45.387460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcf248-042c-4537-abad-25966e215814 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.390060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc1047-7c07-4c93-93b8-0b699e8a1799 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.400118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc1047-7c07-4c93-93b8-0b699e8a1799 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.454102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcf248-042c-4537-abad-25966e215814 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.458999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b03815-d482-4cb3-8356-399c7bd436b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.459706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b03815-d482-4cb3-8356-399c7bd436b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.459994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b03815-d482-4cb3-8356-399c7bd436b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.460274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b03815-d482-4cb3-8356-399c7bd436b4 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.460963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b03815-d482-4cb3-8356-399c7bd436b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.462926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b03815-d482-4cb3-8356-399c7bd436b4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.465787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579ea7c-5b8b-45b4-95f6-23c204a14591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.466685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579ea7c-5b8b-45b4-95f6-23c204a14591 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.467003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579ea7c-5b8b-45b4-95f6-23c204a14591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.467309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579ea7c-5b8b-45b4-95f6-23c204a14591 admin admin] PATH_INFO: `/v3/projects/ba8403a33b7e408891c457d214037b69/users/cb46a242c0064d60aa125f436e72c28e/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.484408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579ea7c-5b8b-45b4-95f6-23c204a14591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba8403a33b7e408891c457d214037b69, user_id=cb46a242c0064d60aa125f436e72c28e, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.488634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 947/1877] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:50:45 2022] POST /identity/v3/users => generated 310 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:50:45.488780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579ea7c-5b8b-45b4-95f6-23c204a14591 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.489669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b03815-d482-4cb3-8356-399c7bd436b4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.500297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b03815-d482-4cb3-8356-399c7bd436b4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.501422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 931/1878] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:45 2022] PUT /identity/v3/projects/ba8403a33b7e408891c457d214037b69/users/cb46a242c0064d60aa125f436e72c28e/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:45.505611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579ea7c-5b8b-45b4-95f6-23c204a14591 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.524651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:45.565485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd6ff2-16d2-4ae3-9373-f499b5179497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.566031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd6ff2-16d2-4ae3-9373-f499b5179497 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.566317 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd6ff2-16d2-4ae3-9373-f499b5179497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.566606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd6ff2-16d2-4ae3-9373-f499b5179497 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.567381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd6ff2-16d2-4ae3-9373-f499b5179497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.568980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd6ff2-16d2-4ae3-9373-f499b5179497 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.570497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579ea7c-5b8b-45b4-95f6-23c204a14591 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.579524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761f332-8731-42f6-bff2-ec19985057b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.580217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761f332-8731-42f6-bff2-ec19985057b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.580508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761f332-8731-42f6-bff2-ec19985057b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.580787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761f332-8731-42f6-bff2-ec19985057b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.581037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 948/1879] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:45.581530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761f332-8731-42f6-bff2-ec19985057b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.583377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761f332-8731-42f6-bff2-ec19985057b3 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.585133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd6ff2-16d2-4ae3-9373-f499b5179497 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.596299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd6ff2-16d2-4ae3-9373-f499b5179497 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.604817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc29d4-63dc-46aa-a28f-b62e14451862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.605937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc29d4-63dc-46aa-a28f-b62e14451862 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.606353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc29d4-63dc-46aa-a28f-b62e14451862 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.606784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc29d4-63dc-46aa-a28f-b62e14451862 admin admin] PATH_INFO: `/v3/projects/3ba38966bdca422ab28ac3b5e24ce07d/users/aeeeda4510c94720a0d673aa0943604b/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.610039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 932/1880] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:50:45.616616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761f332-8731-42f6-bff2-ec19985057b3 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.628180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc29d4-63dc-46aa-a28f-b62e14451862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba38966bdca422ab28ac3b5e24ce07d, user_id=aeeeda4510c94720a0d673aa0943604b, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.630773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc29d4-63dc-46aa-a28f-b62e14451862 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.631012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761f332-8731-42f6-bff2-ec19985057b3 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.636283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e484c-05f1-4c08-9904-19ff778307ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.637257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e484c-05f1-4c08-9904-19ff778307ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.637702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e484c-05f1-4c08-9904-19ff778307ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.637994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e484c-05f1-4c08-9904-19ff778307ea admin admin] PATH_INFO: `/v3/projects/ba8403a33b7e408891c457d214037b69/users/cb46a242c0064d60aa125f436e72c28e/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.644558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e484c-05f1-4c08-9904-19ff778307ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba8403a33b7e408891c457d214037b69, user_id=cb46a242c0064d60aa125f436e72c28e, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.647022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 949/1881] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:45 2022] PUT /identity/v3/projects/3ba38966bdca422ab28ac3b5e24ce07d/users/aeeeda4510c94720a0d673aa0943604b/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:50:45.649289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e484c-05f1-4c08-9904-19ff778307ea admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.650974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc29d4-63dc-46aa-a28f-b62e14451862 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.661128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 933/1882] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:45 2022] PUT /identity/v3/projects/ba8403a33b7e408891c457d214037b69/users/cb46a242c0064d60aa125f436e72c28e/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:45.673038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1a87d-8be6-4e2b-ad36-dc3a4fec4bc9 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.673331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1a87d-8be6-4e2b-ad36-dc3a4fec4bc9 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.673709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1a87d-8be6-4e2b-ad36-dc3a4fec4bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.704261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:45.707851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e1a1a87d-8be6-4e2b-ad36-dc3a4fec4bc9 None None] MFA Rules not processed for user `cb46a242c0064d60aa125f436e72c28e`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:45.709158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e1a1a87d-8be6-4e2b-ad36-dc3a4fec4bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:45.728074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc29d4-63dc-46aa-a28f-b62e14451862 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.805059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955ea15-3ab2-4dc4-b37b-c2ebc0f4befa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.805859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955ea15-3ab2-4dc4-b37b-c2ebc0f4befa admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.806202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955ea15-3ab2-4dc4-b37b-c2ebc0f4befa admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.806568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955ea15-3ab2-4dc4-b37b-c2ebc0f4befa admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.807403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955ea15-3ab2-4dc4-b37b-c2ebc0f4befa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.809096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955ea15-3ab2-4dc4-b37b-c2ebc0f4befa admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.811175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 934/1883] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:50:45.823232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 950/1884] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:50:45.824131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1a87d-8be6-4e2b-ad36-dc3a4fec4bc9 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.825573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e1a1a87d-8be6-4e2b-ad36-dc3a4fec4bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:45.827737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955ea15-3ab2-4dc4-b37b-c2ebc0f4befa admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:45.842126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955ea15-3ab2-4dc4-b37b-c2ebc0f4befa admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.849181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29f19c-b642-4756-b36d-8acbac9d2567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.850430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f19c-b642-4756-b36d-8acbac9d2567 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.850812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f19c-b642-4756-b36d-8acbac9d2567 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.851201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f19c-b642-4756-b36d-8acbac9d2567 admin admin] PATH_INFO: `/v3/projects/3ba38966bdca422ab28ac3b5e24ce07d/users/aeeeda4510c94720a0d673aa0943604b/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.853247 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29f19c-b642-4756-b36d-8acbac9d2567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba38966bdca422ab28ac3b5e24ce07d, user_id=aeeeda4510c94720a0d673aa0943604b, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:45.857046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29f19c-b642-4756-b36d-8acbac9d2567 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:45.869138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 951/1885] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:50:45 2022] PUT /identity/v3/projects/3ba38966bdca422ab28ac3b5e24ce07d/users/aeeeda4510c94720a0d673aa0943604b/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:50:45.879793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da0e7e-08b5-4be4-861a-9c73f3864e56 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.880000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da0e7e-08b5-4be4-861a-9c73f3864e56 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.880268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da0e7e-08b5-4be4-861a-9c73f3864e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:45.894983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1a87d-8be6-4e2b-ad36-dc3a4fec4bc9 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:45.917476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-55da0e7e-08b5-4be4-861a-9c73f3864e56 None None] MFA Rules not processed for user `aeeeda4510c94720a0d673aa0943604b`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:50:45.918595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-55da0e7e-08b5-4be4-861a-9c73f3864e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:45.946008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:45.958829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf8734-5be1-468c-8729-245973427593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:45.959560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf8734-5be1-468c-8729-245973427593 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:45.959838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf8734-5be1-468c-8729-245973427593 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:45.960164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf8734-5be1-468c-8729-245973427593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:46.002424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 952/1886] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:50:46.032477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf8734-5be1-468c-8729-245973427593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:46.033944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf8734-5be1-468c-8729-245973427593 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:46.044127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 935/1887] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:50:46.069586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da0e7e-08b5-4be4-861a-9c73f3864e56 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:46.079476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da0e7e-08b5-4be4-861a-9c73f3864e56 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:46.083420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e2540-924c-497a-a5d3-fb62bfeda99a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:46.084104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e2540-924c-497a-a5d3-fb62bfeda99a service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:46.084394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e2540-924c-497a-a5d3-fb62bfeda99a service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:46.086551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e2540-924c-497a-a5d3-fb62bfeda99a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:46.095063 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e2540-924c-497a-a5d3-fb62bfeda99a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:46.097286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e2540-924c-497a-a5d3-fb62bfeda99a service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:46.126099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 953/1888] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:46 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:46.364274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf8734-5be1-468c-8729-245973427593 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:46.365775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3dbf8734-5be1-468c-8729-245973427593 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:50:46.384091 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf8734-5be1-468c-8729-245973427593 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:46.388142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766a0ab-f785-47f9-96ac-7c211f09159a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:46.389161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766a0ab-f785-47f9-96ac-7c211f09159a service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:46.389611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766a0ab-f785-47f9-96ac-7c211f09159a service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:46.390769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766a0ab-f785-47f9-96ac-7c211f09159a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:46.470415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e2540-924c-497a-a5d3-fb62bfeda99a service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:50:46.482846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e2540-924c-497a-a5d3-fb62bfeda99a service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:50:46.488090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6e94a-6aaf-4b43-b9a5-ec59b78672e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:50:46.488781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e94a-6aaf-4b43-b9a5-ec59b78672e9 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:50:46.489096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e94a-6aaf-4b43-b9a5-ec59b78672e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:50:46.489375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e94a-6aaf-4b43-b9a5-ec59b78672e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:50:46.507894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766a0ab-f785-47f9-96ac-7c211f09159a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:46.509495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766a0ab-f785-47f9-96ac-7c211f09159a service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:46.519657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 936/1889] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:50:46.541252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:46.562896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6e94a-6aaf-4b43-b9a5-ec59b78672e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:50:46.564898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6e94a-6aaf-4b43-b9a5-ec59b78672e9 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:50:46.581755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 954/1890] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:50:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:50:56.515607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=59 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:50:56.573473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=7 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:01.826410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766a0ab-f785-47f9-96ac-7c211f09159a service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:01.905580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766a0ab-f785-47f9-96ac-7c211f09159a service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:01.912645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:01.913745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:01.914199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:01.914482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] PATH_INFO: `/v3/users/651249ee12184319b66f74d3652648f6` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:01.915738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=651249ee12184319b66f74d3652648f6)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:01.917658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:01.982799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '651249ee12184319b66f74d3652648f6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:01.993362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '651249ee12184319b66f74d3652648f6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:02.003629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '651249ee12184319b66f74d3652648f6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:02.004278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '651249ee12184319b66f74d3652648f6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:02.004921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '651249ee12184319b66f74d3652648f6'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:02.039139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eb6f6-129a-4034-bdbd-4d951c60670f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:02.039631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eb6f6-129a-4034-bdbd-4d951c60670f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:02.039709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eb6f6-129a-4034-bdbd-4d951c60670f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:02.046924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 937/1891] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:01 2022] DELETE /identity/v3/users/651249ee12184319b66f74d3652648f6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:02.060521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2e4eb6f6-129a-4034-bdbd-4d951c60670f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:02.061775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2e4eb6f6-129a-4034-bdbd-4d951c60670f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:02.320227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 955/1892] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:02.330598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:02.331573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:02.398204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023db712-856f-4b09-b38a-12e33abf5ec1 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:02.402100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b181926-40b9-489b-9d2c-c5611f284845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:02.403145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b181926-40b9-489b-9d2c-c5611f284845 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:02.403432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b181926-40b9-489b-9d2c-c5611f284845 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:02.403720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b181926-40b9-489b-9d2c-c5611f284845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:02.418936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b181926-40b9-489b-9d2c-c5611f284845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:02.420496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b181926-40b9-489b-9d2c-c5611f284845 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:02.430383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 938/1893] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:02.813095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eb6f6-129a-4034-bdbd-4d951c60670f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:02.816943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2e4eb6f6-129a-4034-bdbd-4d951c60670f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:02.833587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eb6f6-129a-4034-bdbd-4d951c60670f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:02.844956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:02.845848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:02.846129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:02.846403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] PATH_INFO: `/v3/projects/1d9de3185e48434ea92808b08bae27ab` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:02.847934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9de3185e48434ea92808b08bae27ab)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:02.849840 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:02.929598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9de3185e48434ea92808b08bae27ab', 'request_id': 'req-d043d40d-c656-403a-a3b2-4198683d6896'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:02.936683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9de3185e48434ea92808b08bae27ab', 'request_id': 'req-d043d40d-c656-403a-a3b2-4198683d6896'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:02.946987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9de3185e48434ea92808b08bae27ab', 'request_id': 'req-d043d40d-c656-403a-a3b2-4198683d6896'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:02.962208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] The token cache is being invalidate because project 1d9de3185e48434ea92808b08bae27ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:02.962762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:02.965646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:02.969808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 956/1894] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:02 2022] DELETE /identity/v3/projects/1d9de3185e48434ea92808b08bae27ab => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:02.970338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b181926-40b9-489b-9d2c-c5611f284845 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:02.980792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b181926-40b9-489b-9d2c-c5611f284845 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:03.005073 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:03.059274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:03.060515 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:03.060802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:03.061079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] PATH_INFO: `/v3/users/cb46a242c0064d60aa125f436e72c28e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:03.062987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb46a242c0064d60aa125f436e72c28e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:03.064899 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:03.128037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb46a242c0064d60aa125f436e72c28e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.151840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb46a242c0064d60aa125f436e72c28e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.174612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb46a242c0064d60aa125f436e72c28e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.175325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb46a242c0064d60aa125f436e72c28e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.178905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0ecf5344-1c8e-445e-bdb0-fc3d6ce9790e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb46a242c0064d60aa125f436e72c28e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.206848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 939/1895] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:02 2022] DELETE /identity/v3/users/cb46a242c0064d60aa125f436e72c28e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:03.325467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:03.327756 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:03.356212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043d40d-c656-403a-a3b2-4198683d6896 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:03.452049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:03.453794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:03.454752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:03.455790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] PATH_INFO: `/v3/users/41ba7fe63355423e8438d0d1237e983a` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:03.457856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ba7fe63355423e8438d0d1237e983a)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:03.460817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:03.507912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:03.535204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ba7fe63355423e8438d0d1237e983a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.545684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ba7fe63355423e8438d0d1237e983a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.555094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ba7fe63355423e8438d0d1237e983a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.555942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ba7fe63355423e8438d0d1237e983a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.572437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ba7fe63355423e8438d0d1237e983a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.579649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 957/1896] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:03 2022] DELETE /identity/v3/users/41ba7fe63355423e8438d0d1237e983a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:03.585178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6419d5-acaa-4d43-9d21-836367b0570f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:03.585471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6419d5-acaa-4d43-9d21-836367b0570f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:03.585761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6419d5-acaa-4d43-9d21-836367b0570f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:03.606946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-5c6419d5-acaa-4d43-9d21-836367b0570f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:03.608260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5c6419d5-acaa-4d43-9d21-836367b0570f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:03.628791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:03.630507 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:03.734511 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322132b8-fbda-4ad3-aa4e-cc79d81f90f0 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:03.739089 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 940/1897] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:03.751008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6419d5-acaa-4d43-9d21-836367b0570f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:03.752594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5c6419d5-acaa-4d43-9d21-836367b0570f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:03.755187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:03.756086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:03.756363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:03.756645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] PATH_INFO: `/v3/projects/ba8403a33b7e408891c457d214037b69` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:03.758380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba8403a33b7e408891c457d214037b69)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:03.760482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:03.831105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:03.844281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba8403a33b7e408891c457d214037b69', 'request_id': 'req-9e526772-f459-4b43-b006-2686cdfda9ab'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.851143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba8403a33b7e408891c457d214037b69', 'request_id': 'req-9e526772-f459-4b43-b006-2686cdfda9ab'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.854098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6419d5-acaa-4d43-9d21-836367b0570f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:03.862496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba8403a33b7e408891c457d214037b69', 'request_id': 'req-9e526772-f459-4b43-b006-2686cdfda9ab'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.881996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] The token cache is being invalidate because project ba8403a33b7e408891c457d214037b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:03.882296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.883355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e526772-f459-4b43-b006-2686cdfda9ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:03.888435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 958/1898] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:03 2022] DELETE /identity/v3/projects/ba8403a33b7e408891c457d214037b69 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:03.891054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ba149-a6c8-49c3-83bd-7f2992da6c4f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:03.891477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ba149-a6c8-49c3-83bd-7f2992da6c4f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:03.891930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ba149-a6c8-49c3-83bd-7f2992da6c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:03.917032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-834ba149-a6c8-49c3-83bd-7f2992da6c4f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:03.918281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-834ba149-a6c8-49c3-83bd-7f2992da6c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:03.940713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee247d-62e5-473c-9efd-53e07d141a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:03.942118 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee247d-62e5-473c-9efd-53e07d141a3b service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:03.943671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee247d-62e5-473c-9efd-53e07d141a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:03.943981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee247d-62e5-473c-9efd-53e07d141a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:03.946748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-13ee247d-62e5-473c-9efd-53e07d141a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:03.954161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:03.997626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 959/1899] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:04.005779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d251e0a-ed5b-41b8-a985-2dd1e2fabcf0 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.006207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d251e0a-ed5b-41b8-a985-2dd1e2fabcf0 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.015961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d251e0a-ed5b-41b8-a985-2dd1e2fabcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.037152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee247d-62e5-473c-9efd-53e07d141a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.038581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee247d-62e5-473c-9efd-53e07d141a3b service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.039026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-1d251e0a-ed5b-41b8-a985-2dd1e2fabcf0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:04.040288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1d251e0a-ed5b-41b8-a985-2dd1e2fabcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:04.048336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 960/1900] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:04.048764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 941/1901] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:04.052419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d251e0a-ed5b-41b8-a985-2dd1e2fabcf0 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.061487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d251e0a-ed5b-41b8-a985-2dd1e2fabcf0 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.066953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2150f-ede8-45f3-8933-8bb33ad81d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.067658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2150f-ede8-45f3-8933-8bb33ad81d54 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.067941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2150f-ede8-45f3-8933-8bb33ad81d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.068217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2150f-ede8-45f3-8933-8bb33ad81d54 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.068938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2150f-ede8-45f3-8933-8bb33ad81d54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:04.069242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2150f-ede8-45f3-8933-8bb33ad81d54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.070901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2150f-ede8-45f3-8933-8bb33ad81d54 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.084262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 961/1902] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:04.088923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee247d-62e5-473c-9efd-53e07d141a3b service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.099295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee247d-62e5-473c-9efd-53e07d141a3b service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.104447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd28a5-36f0-42e7-b943-e04346d64105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.105165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd28a5-36f0-42e7-b943-e04346d64105 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.105465 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd28a5-36f0-42e7-b943-e04346d64105 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.105750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd28a5-36f0-42e7-b943-e04346d64105 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.106450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd28a5-36f0-42e7-b943-e04346d64105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.108366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd28a5-36f0-42e7-b943-e04346d64105 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.135346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 942/1903] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:04 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:04.139491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2150f-ede8-45f3-8933-8bb33ad81d54 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.150297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2150f-ede8-45f3-8933-8bb33ad81d54 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.214934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776c006-bd5c-4bd8-94aa-3721754a2726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.215615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776c006-bd5c-4bd8-94aa-3721754a2726 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.215899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776c006-bd5c-4bd8-94aa-3721754a2726 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.216175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776c006-bd5c-4bd8-94aa-3721754a2726 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.216861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776c006-bd5c-4bd8-94aa-3721754a2726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.218833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776c006-bd5c-4bd8-94aa-3721754a2726 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.239531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 962/1904] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:04 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:04.243644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd28a5-36f0-42e7-b943-e04346d64105 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.254699 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd28a5-36f0-42e7-b943-e04346d64105 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.259674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4bf06-13d8-444e-856d-c8cb2fa7fd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.260342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4bf06-13d8-444e-856d-c8cb2fa7fd22 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.260625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4bf06-13d8-444e-856d-c8cb2fa7fd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.260902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4bf06-13d8-444e-856d-c8cb2fa7fd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.261657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4bf06-13d8-444e-856d-c8cb2fa7fd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.263269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4bf06-13d8-444e-856d-c8cb2fa7fd22 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.276194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 943/1905] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:04.280109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776c006-bd5c-4bd8-94aa-3721754a2726 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.291051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776c006-bd5c-4bd8-94aa-3721754a2726 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.296659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ddfb8-684c-4dc1-b068-dbdb4601e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.297324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ddfb8-684c-4dc1-b068-dbdb4601e906 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.297609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ddfb8-684c-4dc1-b068-dbdb4601e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.298013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ddfb8-684c-4dc1-b068-dbdb4601e906 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.298817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ddfb8-684c-4dc1-b068-dbdb4601e906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.300544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ddfb8-684c-4dc1-b068-dbdb4601e906 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.314448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 963/1906] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:04.315739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4bf06-13d8-444e-856d-c8cb2fa7fd22 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.326856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4bf06-13d8-444e-856d-c8cb2fa7fd22 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.332365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195a552-f271-47bc-8217-247cec0d36f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.333371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195a552-f271-47bc-8217-247cec0d36f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.333688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195a552-f271-47bc-8217-247cec0d36f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.334016 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195a552-f271-47bc-8217-247cec0d36f9 admin admin] PATH_INFO: `/v3/projects/452f47b13af24caf979895f41f240e48/users/745389010cfa48e18e01489a9b664e95/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.347728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ddfb8-684c-4dc1-b068-dbdb4601e906 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.349522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-dc5ddfb8-684c-4dc1-b068-dbdb4601e906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:04.349835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195a552-f271-47bc-8217-247cec0d36f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=452f47b13af24caf979895f41f240e48, user_id=745389010cfa48e18e01489a9b664e95, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.352614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195a552-f271-47bc-8217-247cec0d36f9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.366549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ddfb8-684c-4dc1-b068-dbdb4601e906 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.368676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 944/1907] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:04 2022] PUT /identity/v3/projects/452f47b13af24caf979895f41f240e48/users/745389010cfa48e18e01489a9b664e95/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:04.379610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea370a78-6fcd-4cda-87d4-8fdaad60cccd None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.379908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea370a78-6fcd-4cda-87d4-8fdaad60cccd None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.380193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea370a78-6fcd-4cda-87d4-8fdaad60cccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.414114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ea370a78-6fcd-4cda-87d4-8fdaad60cccd None None] MFA Rules not processed for user `745389010cfa48e18e01489a9b664e95`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:04.415610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ea370a78-6fcd-4cda-87d4-8fdaad60cccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:04.441155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.442505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.442824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.443108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] PATH_INFO: `/v3/projects/73977e69fd104c9285d2741e0bae8fa1` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.445721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73977e69fd104c9285d2741e0bae8fa1)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.447925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.484347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:04.523220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:04.536433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 945/1908] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:04.541207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea370a78-6fcd-4cda-87d4-8fdaad60cccd None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.541659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73977e69fd104c9285d2741e0bae8fa1', 'request_id': 'req-908ac0e3-fd1f-4ee3-8d5a-816749772619'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.552674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73977e69fd104c9285d2741e0bae8fa1', 'request_id': 'req-908ac0e3-fd1f-4ee3-8d5a-816749772619'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.557278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea370a78-6fcd-4cda-87d4-8fdaad60cccd None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.566101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73977e69fd104c9285d2741e0bae8fa1', 'request_id': 'req-908ac0e3-fd1f-4ee3-8d5a-816749772619'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.587021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] The token cache is being invalidate because project 73977e69fd104c9285d2741e0bae8fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:04.587319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.588742 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.591065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 964/1909] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:04 2022] DELETE /identity/v3/projects/73977e69fd104c9285d2741e0bae8fa1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:04.595913 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.597617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:04.667679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df20d6-b32c-4c18-8dde-ba03c1d34bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.669209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df20d6-b32c-4c18-8dde-ba03c1d34bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.669602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df20d6-b32c-4c18-8dde-ba03c1d34bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.670019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df20d6-b32c-4c18-8dde-ba03c1d34bf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.671074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df20d6-b32c-4c18-8dde-ba03c1d34bf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.674074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df20d6-b32c-4c18-8dde-ba03c1d34bf1 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.681881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ac0e3-fd1f-4ee3-8d5a-816749772619 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.688340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.703285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.703692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.707208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] PATH_INFO: `/v3/users/94cffdd804e448dbb91a375addfa01d3` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.708849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94cffdd804e448dbb91a375addfa01d3)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.711351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 946/1910] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:04 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:04.712331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df20d6-b32c-4c18-8dde-ba03c1d34bf1 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.723862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-94df20d6-b32c-4c18-8dde-ba03c1d34bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:04.735240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.807206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df20d6-b32c-4c18-8dde-ba03c1d34bf1 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:04.815194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:04.816489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.816942 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.817459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] PATH_INFO: `/v3/users/9ba081cb43674c26947b05412d9184f8` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:04.819152 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba081cb43674c26947b05412d9184f8)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:04.821531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:04.856645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94cffdd804e448dbb91a375addfa01d3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.869297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94cffdd804e448dbb91a375addfa01d3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.884375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94cffdd804e448dbb91a375addfa01d3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.885291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94cffdd804e448dbb91a375addfa01d3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.912216 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba081cb43674c26947b05412d9184f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.914611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94cffdd804e448dbb91a375addfa01d3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.922212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba081cb43674c26947b05412d9184f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.930775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 965/1911] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:04 2022] DELETE /identity/v3/users/94cffdd804e448dbb91a375addfa01d3 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:04.930775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:04.936524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba081cb43674c26947b05412d9184f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.937969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba081cb43674c26947b05412d9184f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.938386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:04.938886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-5d432552-8d41-4373-9fe8-0347cb9d2599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba081cb43674c26947b05412d9184f8'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:04.945761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:04.972709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 947/1912] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:04 2022] DELETE /identity/v3/users/9ba081cb43674c26947b05412d9184f8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:04.977949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd7f93-0221-4f18-83cf-db9d028b027c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:04.978366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd7f93-0221-4f18-83cf-db9d028b027c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:04.978859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd7f93-0221-4f18-83cf-db9d028b027c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.007529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-37dd7f93-0221-4f18-83cf-db9d028b027c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:05.008949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-37dd7f93-0221-4f18-83cf-db9d028b027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:05.054536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:05.102157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45632e39-c722-49d4-83ca-5b32dc6eead2 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.130084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 948/1913] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:05.146564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd7f93-0221-4f18-83cf-db9d028b027c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:05.148514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-37dd7f93-0221-4f18-83cf-db9d028b027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:05.183759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb58f5-ef41-4dbf-99f1-c944b4dbf746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:05.184002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb58f5-ef41-4dbf-99f1-c944b4dbf746 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.184254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb58f5-ef41-4dbf-99f1-c944b4dbf746 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.184534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb58f5-ef41-4dbf-99f1-c944b4dbf746 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.185228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb58f5-ef41-4dbf-99f1-c944b4dbf746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:05.188294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb58f5-ef41-4dbf-99f1-c944b4dbf746 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:05.218389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 966/1914] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:04 2022] POST /identity/v3/users => generated 290 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:05.228659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb58f5-ef41-4dbf-99f1-c944b4dbf746 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:05.230373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0afb58f5-ef41-4dbf-99f1-c944b4dbf746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:05.246702 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb58f5-ef41-4dbf-99f1-c944b4dbf746 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.254635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08bbe1-4cdf-4606-91d2-1c6c718df065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:05.255461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08bbe1-4cdf-4606-91d2-1c6c718df065 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.257117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08bbe1-4cdf-4606-91d2-1c6c718df065 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.257317 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08bbe1-4cdf-4606-91d2-1c6c718df065 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.258042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08bbe1-4cdf-4606-91d2-1c6c718df065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:05.259962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08bbe1-4cdf-4606-91d2-1c6c718df065 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:05.262916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd7f93-0221-4f18-83cf-db9d028b027c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.271279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc7aa2-5ca1-412a-9081-80720641abb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:05.272218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc7aa2-5ca1-412a-9081-80720641abb0 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.272690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc7aa2-5ca1-412a-9081-80720641abb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.273142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc7aa2-5ca1-412a-9081-80720641abb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.274541 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 967/1915] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:05.279008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08bbe1-4cdf-4606-91d2-1c6c718df065 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:05.292481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08bbe1-4cdf-4606-91d2-1c6c718df065 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.294674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc7aa2-5ca1-412a-9081-80720641abb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:05.297024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc7aa2-5ca1-412a-9081-80720641abb0 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:05.298266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ffc26-8a5b-4a80-bbae-3e59a920551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:05.299096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ffc26-8a5b-4a80-bbae-3e59a920551a admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.299482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ffc26-8a5b-4a80-bbae-3e59a920551a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.299862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ffc26-8a5b-4a80-bbae-3e59a920551a admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.300706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ffc26-8a5b-4a80-bbae-3e59a920551a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:05.302482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ffc26-8a5b-4a80-bbae-3e59a920551a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:05.315171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 949/1916] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:05.318776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 968/1917] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:05.321726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc7aa2-5ca1-412a-9081-80720641abb0 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:05.331821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc7aa2-5ca1-412a-9081-80720641abb0 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.336659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cd0dd-646d-4f6e-89e1-feab983c1fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:05.337538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cd0dd-646d-4f6e-89e1-feab983c1fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.337820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cd0dd-646d-4f6e-89e1-feab983c1fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.338098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cd0dd-646d-4f6e-89e1-feab983c1fd9 admin admin] PATH_INFO: `/v3/projects/2f02aa8e85f74e58a9a2642158bbe7de/users/34be39bf3bab4e699f660c0e8523422b/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.363169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cd0dd-646d-4f6e-89e1-feab983c1fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f02aa8e85f74e58a9a2642158bbe7de, user_id=34be39bf3bab4e699f660c0e8523422b, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:05.367764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cd0dd-646d-4f6e-89e1-feab983c1fd9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:05.398758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 950/1918] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:05 2022] PUT /identity/v3/projects/2f02aa8e85f74e58a9a2642158bbe7de/users/34be39bf3bab4e699f660c0e8523422b/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:05.399533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e7bee-b857-42b4-bdb0-b9f0f0c0a075 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.399995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e7bee-b857-42b4-bdb0-b9f0f0c0a075 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.400088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e7bee-b857-42b4-bdb0-b9f0f0c0a075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.441032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5a8e7bee-b857-42b4-bdb0-b9f0f0c0a075 None None] MFA Rules not processed for user `34be39bf3bab4e699f660c0e8523422b`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:05.442337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5a8e7bee-b857-42b4-bdb0-b9f0f0c0a075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:05.533826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 969/1919] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:05.537814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cd0dd-646d-4f6e-89e1-feab983c1fd9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:05.660542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cd0dd-646d-4f6e-89e1-feab983c1fd9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.667100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57334d66-b3a9-4743-b9ee-2f92259d4f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:05.668356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57334d66-b3a9-4743-b9ee-2f92259d4f54 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.668356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57334d66-b3a9-4743-b9ee-2f92259d4f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.668616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57334d66-b3a9-4743-b9ee-2f92259d4f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.670349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57334d66-b3a9-4743-b9ee-2f92259d4f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:05.671792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57334d66-b3a9-4743-b9ee-2f92259d4f54 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:05.683351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:05.708742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 951/1920] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:05 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:05.710022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e7bee-b857-42b4-bdb0-b9f0f0c0a075 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:05.721305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57334d66-b3a9-4743-b9ee-2f92259d4f54 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:05.726566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e7bee-b857-42b4-bdb0-b9f0f0c0a075 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.727144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-57334d66-b3a9-4743-b9ee-2f92259d4f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:05.791856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:05.828841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bbbe1-1097-43e9-9012-3addb1740e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:05.832824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bbbe1-1097-43e9-9012-3addb1740e89 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.833326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bbbe1-1097-43e9-9012-3addb1740e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.833793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bbbe1-1097-43e9-9012-3addb1740e89 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.834665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6bbbe1-1097-43e9-9012-3addb1740e89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:05.838634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6bbbe1-1097-43e9-9012-3addb1740e89 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:05.864639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57334d66-b3a9-4743-b9ee-2f92259d4f54 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.870503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:05.871599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.872120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.873414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] PATH_INFO: `/v3/projects/6b39759a4340420886bd14c137162e29` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.874828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 970/1921] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:05 2022] POST /identity/v3/users => generated 296 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:05.875305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b39759a4340420886bd14c137162e29)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:05.876818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bbbe1-1097-43e9-9012-3addb1740e89 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:05.877509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:05.892151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bbbe1-1097-43e9-9012-3addb1740e89 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.898649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909d0bb-593f-4017-a630-7a60bea87ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:05.900083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909d0bb-593f-4017-a630-7a60bea87ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:05.900450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909d0bb-593f-4017-a630-7a60bea87ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:05.901010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909d0bb-593f-4017-a630-7a60bea87ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:05.902142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909d0bb-593f-4017-a630-7a60bea87ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:05.904801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909d0bb-593f-4017-a630-7a60bea87ac6 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:05.920759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 971/1922] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:05.925701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909d0bb-593f-4017-a630-7a60bea87ac6 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:05.936106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909d0bb-593f-4017-a630-7a60bea87ac6 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:05.948009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b39759a4340420886bd14c137162e29', 'request_id': 'req-a76da23b-7dd1-4329-89f8-06a14bb1acda'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:05.959013 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b39759a4340420886bd14c137162e29', 'request_id': 'req-a76da23b-7dd1-4329-89f8-06a14bb1acda'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:05.972891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b39759a4340420886bd14c137162e29', 'request_id': 'req-a76da23b-7dd1-4329-89f8-06a14bb1acda'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:05.997876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] The token cache is being invalidate because project 6b39759a4340420886bd14c137162e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:05.998226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:06.001671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a76da23b-7dd1-4329-89f8-06a14bb1acda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:06.006229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 952/1923] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:05 2022] DELETE /identity/v3/projects/6b39759a4340420886bd14c137162e29 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:06.012686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b527842-eb07-413e-b119-6c6894a5b5c7 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.013216 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b527842-eb07-413e-b119-6c6894a5b5c7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.013216 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b527842-eb07-413e-b119-6c6894a5b5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.025625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d825a9e-943c-4f71-bdcc-e840c453f497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.028289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d825a9e-943c-4f71-bdcc-e840c453f497 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.028805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d825a9e-943c-4f71-bdcc-e840c453f497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.029769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d825a9e-943c-4f71-bdcc-e840c453f497 admin admin] PATH_INFO: `/v3/projects/c09096cc2ab54c858667511df6db46c4/users/848f87a819894f7898dbc1b9b196248c/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.035461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-6b527842-eb07-413e-b119-6c6894a5b5c7 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:06.037045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6b527842-eb07-413e-b119-6c6894a5b5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:06.052180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d825a9e-943c-4f71-bdcc-e840c453f497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09096cc2ab54c858667511df6db46c4, user_id=848f87a819894f7898dbc1b9b196248c, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.055757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d825a9e-943c-4f71-bdcc-e840c453f497 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.056930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 953/1924] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:06.058389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a3a3b-4291-4917-884a-4c41bf256ca3 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.058933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a3a3b-4291-4917-884a-4c41bf256ca3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.059220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a3a3b-4291-4917-884a-4c41bf256ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.078505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 972/1925] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:05 2022] PUT /identity/v3/projects/c09096cc2ab54c858667511df6db46c4/users/848f87a819894f7898dbc1b9b196248c/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:06.082491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d825a9e-943c-4f71-bdcc-e840c453f497 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.084796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2d825a9e-943c-4f71-bdcc-e840c453f497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:06.088680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-542a3a3b-4291-4917-884a-4c41bf256ca3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:06.091117 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-542a3a3b-4291-4917-884a-4c41bf256ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:06.183782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 954/1926] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:06.184335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d825a9e-943c-4f71-bdcc-e840c453f497 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.188562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a3a3b-4291-4917-884a-4c41bf256ca3 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.189773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b1d53-0d5b-4db4-b335-9652b44073e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.190615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b1d53-0d5b-4db4-b335-9652b44073e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.191061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b1d53-0d5b-4db4-b335-9652b44073e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.191499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b1d53-0d5b-4db4-b335-9652b44073e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.192442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b1d53-0d5b-4db4-b335-9652b44073e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.195026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b1d53-0d5b-4db4-b335-9652b44073e1 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.200603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a3a3b-4291-4917-884a-4c41bf256ca3 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.218613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 973/1927] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:06.218779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64af80b-1241-4ad5-afa0-d743d4f7d8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.218779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af80b-1241-4ad5-afa0-d743d4f7d8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.218779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af80b-1241-4ad5-afa0-d743d4f7d8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.218779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af80b-1241-4ad5-afa0-d743d4f7d8d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.219123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64af80b-1241-4ad5-afa0-d743d4f7d8d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:06.219632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64af80b-1241-4ad5-afa0-d743d4f7d8d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.220566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b1d53-0d5b-4db4-b335-9652b44073e1 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.221500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64af80b-1241-4ad5-afa0-d743d4f7d8d5 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.234239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:06.238910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b1d53-0d5b-4db4-b335-9652b44073e1 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.243477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 955/1928] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:06.243825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64af80b-1241-4ad5-afa0-d743d4f7d8d5 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.260556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3aee3-9d84-4968-849e-1bd57ed0787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.261466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3aee3-9d84-4968-849e-1bd57ed0787d admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.261748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3aee3-9d84-4968-849e-1bd57ed0787d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.262023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3aee3-9d84-4968-849e-1bd57ed0787d admin admin] PATH_INFO: `/v3/projects/c09096cc2ab54c858667511df6db46c4/users/848f87a819894f7898dbc1b9b196248c/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.270759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3aee3-9d84-4968-849e-1bd57ed0787d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09096cc2ab54c858667511df6db46c4, user_id=848f87a819894f7898dbc1b9b196248c, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.273408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3aee3-9d84-4968-849e-1bd57ed0787d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.284059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64af80b-1241-4ad5-afa0-d743d4f7d8d5 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.297990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509b090-8072-4efc-a21f-233b3df6c742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.298985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509b090-8072-4efc-a21f-233b3df6c742 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.308627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 974/1929] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:06 2022] PUT /identity/v3/projects/c09096cc2ab54c858667511df6db46c4/users/848f87a819894f7898dbc1b9b196248c/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:06.308868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509b090-8072-4efc-a21f-233b3df6c742 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.308868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509b090-8072-4efc-a21f-233b3df6c742 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.308868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509b090-8072-4efc-a21f-233b3df6c742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.308868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509b090-8072-4efc-a21f-233b3df6c742 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.311649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7c3b8-eb6d-42cb-867d-6027a764f64b None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.311884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7c3b8-eb6d-42cb-867d-6027a764f64b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.313444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7c3b8-eb6d-42cb-867d-6027a764f64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.373809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 956/1930] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:06 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:06.377675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509b090-8072-4efc-a21f-233b3df6c742 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.388572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-07e7c3b8-eb6d-42cb-867d-6027a764f64b None None] MFA Rules not processed for user `848f87a819894f7898dbc1b9b196248c`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:06.389834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-07e7c3b8-eb6d-42cb-867d-6027a764f64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:06.429841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:06.473273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509b090-8072-4efc-a21f-233b3df6c742 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.479505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c838ad-13d1-4998-ae70-f295136b0bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.480574 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c838ad-13d1-4998-ae70-f295136b0bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.481325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c838ad-13d1-4998-ae70-f295136b0bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.481864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c838ad-13d1-4998-ae70-f295136b0bf2 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.482841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c838ad-13d1-4998-ae70-f295136b0bf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.486075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c838ad-13d1-4998-ae70-f295136b0bf2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.494585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 975/1931] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:06.510202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7c3b8-eb6d-42cb-867d-6027a764f64b None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.511874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-07e7c3b8-eb6d-42cb-867d-6027a764f64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:06.531936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 957/1932] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:06 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:06.537240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c838ad-13d1-4998-ae70-f295136b0bf2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.564796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c838ad-13d1-4998-ae70-f295136b0bf2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.579179 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da95b81-b3b0-4026-8cf0-c612a910ae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.580202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da95b81-b3b0-4026-8cf0-c612a910ae31 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.581173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da95b81-b3b0-4026-8cf0-c612a910ae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.581677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7c3b8-eb6d-42cb-867d-6027a764f64b None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.582130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da95b81-b3b0-4026-8cf0-c612a910ae31 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.584684 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da95b81-b3b0-4026-8cf0-c612a910ae31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.585860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a2bd5-8717-42f6-9d91-30ba80ddb916 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.586633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a2bd5-8717-42f6-9d91-30ba80ddb916 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.587050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a2bd5-8717-42f6-9d91-30ba80ddb916 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.587423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a2bd5-8717-42f6-9d91-30ba80ddb916 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.589234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da95b81-b3b0-4026-8cf0-c612a910ae31 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.599479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a2bd5-8717-42f6-9d91-30ba80ddb916 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.601104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a2bd5-8717-42f6-9d91-30ba80ddb916 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.620077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 958/1933] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:06.624875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 976/1934] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:06.626410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da95b81-b3b0-4026-8cf0-c612a910ae31 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.646883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da95b81-b3b0-4026-8cf0-c612a910ae31 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.652192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebd992-3f50-49a5-8c5e-ace26c9a6531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.652931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebd992-3f50-49a5-8c5e-ace26c9a6531 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.653226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebd992-3f50-49a5-8c5e-ace26c9a6531 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.653510 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebd992-3f50-49a5-8c5e-ace26c9a6531 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.654262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebd992-3f50-49a5-8c5e-ace26c9a6531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.656754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebd992-3f50-49a5-8c5e-ace26c9a6531 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.669955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a2bd5-8717-42f6-9d91-30ba80ddb916 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.679668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 959/1935] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:06.681087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a2bd5-8717-42f6-9d91-30ba80ddb916 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.683718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebd992-3f50-49a5-8c5e-ace26c9a6531 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.692055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107861cc-8d2f-448b-9942-c2da4bedaed6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.693724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107861cc-8d2f-448b-9942-c2da4bedaed6 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.694462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107861cc-8d2f-448b-9942-c2da4bedaed6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.694954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107861cc-8d2f-448b-9942-c2da4bedaed6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.699221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-107861cc-8d2f-448b-9942-c2da4bedaed6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:06.700688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebd992-3f50-49a5-8c5e-ace26c9a6531 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.707725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a417f5-f321-4f94-9dfe-516ee1c986e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:06.709735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a417f5-f321-4f94-9dfe-516ee1c986e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.710404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a417f5-f321-4f94-9dfe-516ee1c986e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.710931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a417f5-f321-4f94-9dfe-516ee1c986e0 admin admin] PATH_INFO: `/v3/projects/0a811157921b4996bb7483fa4edb02e3/users/d894f86397694569b8aefc0953803f96/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.735760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a417f5-f321-4f94-9dfe-516ee1c986e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a811157921b4996bb7483fa4edb02e3, user_id=d894f86397694569b8aefc0953803f96, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.739742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a417f5-f321-4f94-9dfe-516ee1c986e0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.764354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 960/1936] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:06 2022] PUT /identity/v3/projects/0a811157921b4996bb7483fa4edb02e3/users/d894f86397694569b8aefc0953803f96/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:06.776505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3518a-eb77-4ad7-a904-397c3981daf0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:06.777005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3518a-eb77-4ad7-a904-397c3981daf0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:06.777519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3518a-eb77-4ad7-a904-397c3981daf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:06.782436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107861cc-8d2f-448b-9942-c2da4bedaed6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:06.784145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107861cc-8d2f-448b-9942-c2da4bedaed6 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:06.796934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 977/1937] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:06.817416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d0c3518a-eb77-4ad7-a904-397c3981daf0 None None] MFA Rules not processed for user `d894f86397694569b8aefc0953803f96`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:06.820109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d0c3518a-eb77-4ad7-a904-397c3981daf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:06.950598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 961/1938] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:06.955805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107861cc-8d2f-448b-9942-c2da4bedaed6 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.977566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107861cc-8d2f-448b-9942-c2da4bedaed6 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:06.981563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3518a-eb77-4ad7-a904-397c3981daf0 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:06.984175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d0c3518a-eb77-4ad7-a904-397c3981daf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:07.031329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:07.102955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:07.114998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6c387-2566-497d-a54f-eff4eec054d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:07.115670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6c387-2566-497d-a54f-eff4eec054d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:07.115952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6c387-2566-497d-a54f-eff4eec054d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:07.116228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6c387-2566-497d-a54f-eff4eec054d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:07.117275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6c387-2566-497d-a54f-eff4eec054d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:07.121380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6c387-2566-497d-a54f-eff4eec054d5 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:07.170448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3518a-eb77-4ad7-a904-397c3981daf0 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:07.188215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 978/1939] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:06 2022] POST /identity/v3/projects => generated 373 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:07.192107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6c387-2566-497d-a54f-eff4eec054d5 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:07.197575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:07.202303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:07.208600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:07.208600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] PATH_INFO: `/v3/projects/abad183a4e564844952333f4cbecea03` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:07.213305 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] RBAC: Authorizing `identity:delete_project(project_id=abad183a4e564844952333f4cbecea03)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:07.222830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:07.463259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abad183a4e564844952333f4cbecea03', 'request_id': 'req-840fa734-6660-4fbe-bfb2-0c4046ed874e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.464065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6c387-2566-497d-a54f-eff4eec054d5 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:07.485493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abad183a4e564844952333f4cbecea03', 'request_id': 'req-840fa734-6660-4fbe-bfb2-0c4046ed874e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.509850 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abad183a4e564844952333f4cbecea03', 'request_id': 'req-840fa734-6660-4fbe-bfb2-0c4046ed874e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.536233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] The token cache is being invalidate because project abad183a4e564844952333f4cbecea03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:07.536966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.537887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.540256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 962/1940] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:06 2022] DELETE /identity/v3/projects/abad183a4e564844952333f4cbecea03 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:07.543804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:07.548441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:07.640097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af025b7f-6dfe-4909-88c2-9db4c9e36104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:07.641140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af025b7f-6dfe-4909-88c2-9db4c9e36104 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:07.641572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af025b7f-6dfe-4909-88c2-9db4c9e36104 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:07.643555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af025b7f-6dfe-4909-88c2-9db4c9e36104 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:07.644453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af025b7f-6dfe-4909-88c2-9db4c9e36104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:07.649981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af025b7f-6dfe-4909-88c2-9db4c9e36104 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:07.699429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 979/1941] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:07 2022] POST /identity/v3/users => generated 293 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:07.703555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af025b7f-6dfe-4909-88c2-9db4c9e36104 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:07.708265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-af025b7f-6dfe-4909-88c2-9db4c9e36104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:07.725764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fa734-6660-4fbe-bfb2-0c4046ed874e admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:07.735095 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af025b7f-6dfe-4909-88c2-9db4c9e36104 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:07.742192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:07.742919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:07.743202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:07.743479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] PATH_INFO: `/v3/users/aeeeda4510c94720a0d673aa0943604b` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:07.744881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285390b-d2f6-407e-a0ed-de2f1d35fc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:07.745564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285390b-d2f6-407e-a0ed-de2f1d35fc20 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:07.745837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285390b-d2f6-407e-a0ed-de2f1d35fc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:07.747531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeeeda4510c94720a0d673aa0943604b)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:07.748617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285390b-d2f6-407e-a0ed-de2f1d35fc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:07.749213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285390b-d2f6-407e-a0ed-de2f1d35fc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:07.751791 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:07.754581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285390b-d2f6-407e-a0ed-de2f1d35fc20 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:07.786847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 980/1942] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:07.792347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285390b-d2f6-407e-a0ed-de2f1d35fc20 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:07.818920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285390b-d2f6-407e-a0ed-de2f1d35fc20 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:07.835054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680faf4-7c8d-4daf-93da-079f2f2e4c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:07.836405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680faf4-7c8d-4daf-93da-079f2f2e4c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:07.837126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680faf4-7c8d-4daf-93da-079f2f2e4c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:07.837622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680faf4-7c8d-4daf-93da-079f2f2e4c8c admin admin] PATH_INFO: `/v3/projects/8e43d22eb52641489cae18edf992f5af/users/997176e41fbe4d099b6f6d9473feebaa/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:07.865816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:07.871259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680faf4-7c8d-4daf-93da-079f2f2e4c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e43d22eb52641489cae18edf992f5af, user_id=997176e41fbe4d099b6f6d9473feebaa, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:07.875002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680faf4-7c8d-4daf-93da-079f2f2e4c8c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:07.880857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeeeda4510c94720a0d673aa0943604b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.891868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeeeda4510c94720a0d673aa0943604b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.894160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 981/1943] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:07 2022] PUT /identity/v3/projects/8e43d22eb52641489cae18edf992f5af/users/997176e41fbe4d099b6f6d9473feebaa/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:07.898764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680faf4-7c8d-4daf-93da-079f2f2e4c8c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:07.900593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeeeda4510c94720a0d673aa0943604b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.902217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeeeda4510c94720a0d673aa0943604b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.903396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeeeda4510c94720a0d673aa0943604b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:07.956590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 963/1944] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:07 2022] DELETE /identity/v3/users/aeeeda4510c94720a0d673aa0943604b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:07.968802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:07.994229 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680faf4-7c8d-4daf-93da-079f2f2e4c8c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:08.072425 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242bcf0-48a3-463c-a523-33d1539d2ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:08.073667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242bcf0-48a3-463c-a523-33d1539d2ded admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:08.074063 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242bcf0-48a3-463c-a523-33d1539d2ded admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:08.074623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242bcf0-48a3-463c-a523-33d1539d2ded admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:08.075855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242bcf0-48a3-463c-a523-33d1539d2ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:08.077602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242bcf0-48a3-463c-a523-33d1539d2ded admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:08.094830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 982/1945] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:08.097316 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:08.099229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:08.116184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a565fc5-bab2-4ffa-a85a-a5afd7022259 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:08.121670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd9452-8813-4c47-a4a3-f61a24922f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:08.122572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd9452-8813-4c47-a4a3-f61a24922f7d admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:08.122938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd9452-8813-4c47-a4a3-f61a24922f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:08.123213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd9452-8813-4c47-a4a3-f61a24922f7d admin admin] PATH_INFO: `/v3/projects/8e43d22eb52641489cae18edf992f5af/users/997176e41fbe4d099b6f6d9473feebaa/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:08.125972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd9452-8813-4c47-a4a3-f61a24922f7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e43d22eb52641489cae18edf992f5af, user_id=997176e41fbe4d099b6f6d9473feebaa, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:08.128677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd9452-8813-4c47-a4a3-f61a24922f7d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:08.142333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 964/1946] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:08 2022] PUT /identity/v3/projects/8e43d22eb52641489cae18edf992f5af/users/997176e41fbe4d099b6f6d9473feebaa/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:08.154537 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7ed2e-f404-4bdf-8f62-8349b36b35b3 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:08.154887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7ed2e-f404-4bdf-8f62-8349b36b35b3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:08.155176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7ed2e-f404-4bdf-8f62-8349b36b35b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:08.212482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b3c7ed2e-f404-4bdf-8f62-8349b36b35b3 None None] MFA Rules not processed for user `997176e41fbe4d099b6f6d9473feebaa`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:08.214288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b3c7ed2e-f404-4bdf-8f62-8349b36b35b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:08.321244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 983/1947] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:08.329925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd9452-8813-4c47-a4a3-f61a24922f7d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:08.334631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-dbcd9452-8813-4c47-a4a3-f61a24922f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:08.413109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd9452-8813-4c47-a4a3-f61a24922f7d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:08.419828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bfa3b-e428-4df7-910c-660f19bcfe6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:08.420775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bfa3b-e428-4df7-910c-660f19bcfe6a service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:08.421318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bfa3b-e428-4df7-910c-660f19bcfe6a service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:08.421821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bfa3b-e428-4df7-910c-660f19bcfe6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:08.439913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bfa3b-e428-4df7-910c-660f19bcfe6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:08.441689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bfa3b-e428-4df7-910c-660f19bcfe6a service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:08.458272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 965/1948] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:08.518386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7ed2e-f404-4bdf-8f62-8349b36b35b3 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:08.520057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b3c7ed2e-f404-4bdf-8f62-8349b36b35b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:08.630431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7ed2e-f404-4bdf-8f62-8349b36b35b3 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:08.635617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:08.636603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:08.637003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:08.637383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] PATH_INFO: `/v3/projects/3ba38966bdca422ab28ac3b5e24ce07d` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:08.641010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ba38966bdca422ab28ac3b5e24ce07d)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:08.643107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:08.669951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:08.729647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ba38966bdca422ab28ac3b5e24ce07d', 'request_id': 'req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:08.743850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ba38966bdca422ab28ac3b5e24ce07d', 'request_id': 'req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:08.761813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ba38966bdca422ab28ac3b5e24ce07d', 'request_id': 'req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:08.783142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] The token cache is being invalidate because project 3ba38966bdca422ab28ac3b5e24ce07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:08.783594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:08.784277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-bb3c9852-adfe-42f0-8ba2-9cffb8d5ef12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:08.790940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 984/1949] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:08 2022] DELETE /identity/v3/projects/3ba38966bdca422ab28ac3b5e24ce07d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:08.791712 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd7f6d-e91a-4a9c-adee-3cea13abde35 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:08.791996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd7f6d-e91a-4a9c-adee-3cea13abde35 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:08.792270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd7f6d-e91a-4a9c-adee-3cea13abde35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:08.822849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f9cd7f6d-e91a-4a9c-adee-3cea13abde35 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:08.824061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f9cd7f6d-e91a-4a9c-adee-3cea13abde35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:08.910247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 966/1950] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:08.912913 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22274d8c-0895-41b8-b625-dc206a6b29ae None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:08.913202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22274d8c-0895-41b8-b625-dc206a6b29ae None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:08.913480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22274d8c-0895-41b8-b625-dc206a6b29ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:08.933614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-22274d8c-0895-41b8-b625-dc206a6b29ae None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:08.934972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-22274d8c-0895-41b8-b625-dc206a6b29ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:08.943845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 985/1951] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:08.948669 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd7f6d-e91a-4a9c-adee-3cea13abde35 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:08.960835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd7f6d-e91a-4a9c-adee-3cea13abde35 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:08.966920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:08.967541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:08.968117 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:08.968841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:08.972955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:08.972955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:08.972955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:08.989713 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22274d8c-0895-41b8-b625-dc206a6b29ae None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:08.995506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 967/1952] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:51:08.999525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22274d8c-0895-41b8-b625-dc206a6b29ae None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.004568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af6214-63c1-4c3c-9b33-036f1a9f8b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.005497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af6214-63c1-4c3c-9b33-036f1a9f8b8c admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.005882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af6214-63c1-4c3c-9b33-036f1a9f8b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.006281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af6214-63c1-4c3c-9b33-036f1a9f8b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.007130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af6214-63c1-4c3c-9b33-036f1a9f8b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.009313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af6214-63c1-4c3c-9b33-036f1a9f8b8c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.028910 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.031843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:09.047522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690372e6-6248-4a38-bd99-c516571513ab admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.055989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.057041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.057432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.057811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] PATH_INFO: `/v3/users/d894f86397694569b8aefc0953803f96` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.059168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d894f86397694569b8aefc0953803f96)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.061383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.062039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 986/1953] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:08 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:09.066978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af6214-63c1-4c3c-9b33-036f1a9f8b8c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.108250 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:09.132337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd894f86397694569b8aefc0953803f96'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.148270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd894f86397694569b8aefc0953803f96'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.160726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd894f86397694569b8aefc0953803f96'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.162969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd894f86397694569b8aefc0953803f96'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.163706 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0128d5bf-b241-47dd-b367-625803d77700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd894f86397694569b8aefc0953803f96'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.177398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af6214-63c1-4c3c-9b33-036f1a9f8b8c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.183794 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0efcb-55c4-4cf2-a8d4-50b8d238d38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.184686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0efcb-55c4-4cf2-a8d4-50b8d238d38b admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.185149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0efcb-55c4-4cf2-a8d4-50b8d238d38b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.185432 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0efcb-55c4-4cf2-a8d4-50b8d238d38b admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.186129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0efcb-55c4-4cf2-a8d4-50b8d238d38b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.188080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0efcb-55c4-4cf2-a8d4-50b8d238d38b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.190628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 968/1954] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:09 2022] DELETE /identity/v3/users/d894f86397694569b8aefc0953803f96 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:09.195012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662a17-4410-4060-8dc2-6da61148a9cd None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.195836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662a17-4410-4060-8dc2-6da61148a9cd None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.196146 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662a17-4410-4060-8dc2-6da61148a9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.211558 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 987/1955] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:09 2022] POST /identity/v3/users => generated 283 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:09.215185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0efcb-55c4-4cf2-a8d4-50b8d238d38b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.216912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-64d0efcb-55c4-4cf2-a8d4-50b8d238d38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:09.226349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9c662a17-4410-4060-8dc2-6da61148a9cd None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:09.227995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9c662a17-4410-4060-8dc2-6da61148a9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:09.230069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0efcb-55c4-4cf2-a8d4-50b8d238d38b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.305729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572bb47-0f62-46ac-a602-b0569877575b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.306407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572bb47-0f62-46ac-a602-b0569877575b admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.306690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572bb47-0f62-46ac-a602-b0569877575b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.307003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572bb47-0f62-46ac-a602-b0569877575b admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.307979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572bb47-0f62-46ac-a602-b0569877575b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.308840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 969/1956] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:09.309614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572bb47-0f62-46ac-a602-b0569877575b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.319224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662a17-4410-4060-8dc2-6da61148a9cd None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.320620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9c662a17-4410-4060-8dc2-6da61148a9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:09.322379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:09.332706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 988/1957] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:09.334306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572bb47-0f62-46ac-a602-b0569877575b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.346895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572bb47-0f62-46ac-a602-b0569877575b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.351757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb1a9-5ddf-4aa6-ae3e-7dd43e5e364b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.352436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb1a9-5ddf-4aa6-ae3e-7dd43e5e364b admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.352717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb1a9-5ddf-4aa6-ae3e-7dd43e5e364b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.352999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb1a9-5ddf-4aa6-ae3e-7dd43e5e364b admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.353727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1eb1a9-5ddf-4aa6-ae3e-7dd43e5e364b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.355368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1eb1a9-5ddf-4aa6-ae3e-7dd43e5e364b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.366123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 989/1958] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:09.370354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb1a9-5ddf-4aa6-ae3e-7dd43e5e364b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.382009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb1a9-5ddf-4aa6-ae3e-7dd43e5e364b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.386230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662a17-4410-4060-8dc2-6da61148a9cd None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.387418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46088f63-2285-46ff-82df-5f932d8b3337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.388361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46088f63-2285-46ff-82df-5f932d8b3337 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.388640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46088f63-2285-46ff-82df-5f932d8b3337 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.388938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46088f63-2285-46ff-82df-5f932d8b3337 admin admin] PATH_INFO: `/v3/projects/764a223e4ade426ea0ee2c72e6907f63/users/1e513343b44f408ca7d16e019c8a02ce/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.390372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd637b-51a7-4498-b2b6-f68e62c2ce05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.391085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd637b-51a7-4498-b2b6-f68e62c2ce05 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.391359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd637b-51a7-4498-b2b6-f68e62c2ce05 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.391642 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd637b-51a7-4498-b2b6-f68e62c2ce05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.401419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd637b-51a7-4498-b2b6-f68e62c2ce05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.402909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd637b-51a7-4498-b2b6-f68e62c2ce05 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.405352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46088f63-2285-46ff-82df-5f932d8b3337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764a223e4ade426ea0ee2c72e6907f63, user_id=1e513343b44f408ca7d16e019c8a02ce, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.408615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46088f63-2285-46ff-82df-5f932d8b3337 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.413951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 970/1959] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:09.428566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 990/1960] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:09 2022] PUT /identity/v3/projects/764a223e4ade426ea0ee2c72e6907f63/users/1e513343b44f408ca7d16e019c8a02ce/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:09.438952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f16a67-12bd-4cb0-9bb3-c3bd45e580ec None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.439558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f16a67-12bd-4cb0-9bb3-c3bd45e580ec None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.440083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f16a67-12bd-4cb0-9bb3-c3bd45e580ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.478154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-37f16a67-12bd-4cb0-9bb3-c3bd45e580ec None None] MFA Rules not processed for user `1e513343b44f408ca7d16e019c8a02ce`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:09.479737 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-37f16a67-12bd-4cb0-9bb3-c3bd45e580ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:09.540775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:09.573057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 971/1961] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:09.577039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46088f63-2285-46ff-82df-5f932d8b3337 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.654196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46088f63-2285-46ff-82df-5f932d8b3337 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.661241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fea063-2434-4c1b-9373-fc8aa979ac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.662302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fea063-2434-4c1b-9373-fc8aa979ac5b admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.662685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fea063-2434-4c1b-9373-fc8aa979ac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.663099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fea063-2434-4c1b-9373-fc8aa979ac5b admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.663895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fea063-2434-4c1b-9373-fc8aa979ac5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.665945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fea063-2434-4c1b-9373-fc8aa979ac5b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.712169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 991/1962] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:09 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:09.713492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f16a67-12bd-4cb0-9bb3-c3bd45e580ec None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.725639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f16a67-12bd-4cb0-9bb3-c3bd45e580ec None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.732566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fea063-2434-4c1b-9373-fc8aa979ac5b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.734415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c6fea063-2434-4c1b-9373-fc8aa979ac5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:09.797671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a0e85-4d75-413c-b482-017495a63222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.798431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a0e85-4d75-413c-b482-017495a63222 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.798938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a0e85-4d75-413c-b482-017495a63222 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.799290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a0e85-4d75-413c-b482-017495a63222 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.800051 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a0e85-4d75-413c-b482-017495a63222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.802028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a0e85-4d75-413c-b482-017495a63222 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.816966 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fea063-2434-4c1b-9373-fc8aa979ac5b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.821699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.822569 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.822884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.823164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] PATH_INFO: `/v3/projects/0a811157921b4996bb7483fa4edb02e3` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.824373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a811157921b4996bb7483fa4edb02e3)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.825903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 972/1963] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:09 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:09.826289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.830811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a0e85-4d75-413c-b482-017495a63222 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.842392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a0e85-4d75-413c-b482-017495a63222 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.847857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5895959-945d-49e5-9fb8-f8c0305f063d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.848649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5895959-945d-49e5-9fb8-f8c0305f063d admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.849123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5895959-945d-49e5-9fb8-f8c0305f063d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.849334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5895959-945d-49e5-9fb8-f8c0305f063d admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.850072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5895959-945d-49e5-9fb8-f8c0305f063d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.851682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5895959-945d-49e5-9fb8-f8c0305f063d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.867204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 973/1964] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:09.868915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5895959-945d-49e5-9fb8-f8c0305f063d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.882338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5895959-945d-49e5-9fb8-f8c0305f063d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.896149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a811157921b4996bb7483fa4edb02e3', 'request_id': 'req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.899875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:09.912804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a811157921b4996bb7483fa4edb02e3', 'request_id': 'req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.923064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a811157921b4996bb7483fa4edb02e3', 'request_id': 'req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.939623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] The token cache is being invalidate because project 0a811157921b4996bb7483fa4edb02e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:09.939735 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.940528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:09.942238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 992/1965] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:09 2022] DELETE /identity/v3/projects/0a811157921b4996bb7483fa4edb02e3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:09.946454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.947944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:09.952134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53c61d-79bd-4b76-8acd-7a109961c30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.953957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53c61d-79bd-4b76-8acd-7a109961c30b admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.954411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53c61d-79bd-4b76-8acd-7a109961c30b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.954765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53c61d-79bd-4b76-8acd-7a109961c30b admin admin] PATH_INFO: `/v3/projects/e2c6c6ad15f944b6bd7177b99fffa1fa/users/67967fd102a64f7e94821f2e86531eca/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.965065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d79dd-352a-4f86-bf8b-6cb2e3bf35cf admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:09.970858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:09.971753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:09.972028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:09.972308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] PATH_INFO: `/v3/users/997176e41fbe4d099b6f6d9473feebaa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:09.973345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53c61d-79bd-4b76-8acd-7a109961c30b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c6c6ad15f944b6bd7177b99fffa1fa, user_id=67967fd102a64f7e94821f2e86531eca, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.973598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=997176e41fbe4d099b6f6d9473feebaa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:09.975703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.975985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53c61d-79bd-4b76-8acd-7a109961c30b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:09.991405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 974/1966] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:09 2022] PUT /identity/v3/projects/e2c6c6ad15f944b6bd7177b99fffa1fa/users/67967fd102a64f7e94821f2e86531eca/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:09.996359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53c61d-79bd-4b76-8acd-7a109961c30b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:09.998129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fa53c61d-79bd-4b76-8acd-7a109961c30b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:10.035022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '997176e41fbe4d099b6f6d9473feebaa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.046125 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '997176e41fbe4d099b6f6d9473feebaa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.053800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '997176e41fbe4d099b6f6d9473feebaa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.054491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '997176e41fbe4d099b6f6d9473feebaa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.060616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:10.069941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '997176e41fbe4d099b6f6d9473feebaa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.080117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 993/1967] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:09 2022] DELETE /identity/v3/users/997176e41fbe4d099b6f6d9473feebaa => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:10.083255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53c61d-79bd-4b76-8acd-7a109961c30b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:10.157347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6a97a-5435-4d23-a422-734c41ca4e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:10.158297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6a97a-5435-4d23-a422-734c41ca4e05 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.158586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6a97a-5435-4d23-a422-734c41ca4e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.159288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6a97a-5435-4d23-a422-734c41ca4e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.160388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6a97a-5435-4d23-a422-734c41ca4e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:10.161898 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6a97a-5435-4d23-a422-734c41ca4e05 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:10.177246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 975/1968] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:10.178773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:10.180389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:10.203918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90668bcf-a388-4fb7-be18-53c41e344a44 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:10.209191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78f185-3c7d-4b91-8844-a1a39f5e16ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:10.210086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78f185-3c7d-4b91-8844-a1a39f5e16ce admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.210368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78f185-3c7d-4b91-8844-a1a39f5e16ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.210643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78f185-3c7d-4b91-8844-a1a39f5e16ce admin admin] PATH_INFO: `/v3/projects/e2c6c6ad15f944b6bd7177b99fffa1fa/users/67967fd102a64f7e94821f2e86531eca/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.212659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78f185-3c7d-4b91-8844-a1a39f5e16ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c6c6ad15f944b6bd7177b99fffa1fa, user_id=67967fd102a64f7e94821f2e86531eca, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:10.215473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78f185-3c7d-4b91-8844-a1a39f5e16ce admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:10.226842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 994/1969] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:10 2022] PUT /identity/v3/projects/e2c6c6ad15f944b6bd7177b99fffa1fa/users/67967fd102a64f7e94821f2e86531eca/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:10.238089 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffb674-7393-453c-ad9d-b9fd1a41e568 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.238388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffb674-7393-453c-ad9d-b9fd1a41e568 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.238677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffb674-7393-453c-ad9d-b9fd1a41e568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.269584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-beffb674-7393-453c-ad9d-b9fd1a41e568 None None] MFA Rules not processed for user `67967fd102a64f7e94821f2e86531eca`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:10.270835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-beffb674-7393-453c-ad9d-b9fd1a41e568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:10.350485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 976/1970] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:10 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:10.363083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78f185-3c7d-4b91-8844-a1a39f5e16ce admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:10.364746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bb78f185-3c7d-4b91-8844-a1a39f5e16ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:10.405900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffb674-7393-453c-ad9d-b9fd1a41e568 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:10.407487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-beffb674-7393-453c-ad9d-b9fd1a41e568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:10.445258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78f185-3c7d-4b91-8844-a1a39f5e16ce admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:10.449453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0bacd-4ddc-4c0e-8a8d-dfb62bdfab89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:10.450114 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0bacd-4ddc-4c0e-8a8d-dfb62bdfab89 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.450401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0bacd-4ddc-4c0e-8a8d-dfb62bdfab89 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.450679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0bacd-4ddc-4c0e-8a8d-dfb62bdfab89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.462775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0bacd-4ddc-4c0e-8a8d-dfb62bdfab89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:10.464271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0bacd-4ddc-4c0e-8a8d-dfb62bdfab89 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:10.475446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 995/1971] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:10.493882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffb674-7393-453c-ad9d-b9fd1a41e568 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:10.500702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:10.501729 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.502021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.502295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] PATH_INFO: `/v3/projects/8e43d22eb52641489cae18edf992f5af` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.503511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e43d22eb52641489cae18edf992f5af)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:10.505424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:10.533116 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:10.563818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0bacd-4ddc-4c0e-8a8d-dfb62bdfab89 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:10.565740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a7b0bacd-4ddc-4c0e-8a8d-dfb62bdfab89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:10.577925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0bacd-4ddc-4c0e-8a8d-dfb62bdfab89 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:10.580532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e43d22eb52641489cae18edf992f5af', 'request_id': 'req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.588430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e43d22eb52641489cae18edf992f5af', 'request_id': 'req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.598338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e43d22eb52641489cae18edf992f5af', 'request_id': 'req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.610823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] The token cache is being invalidate because project 8e43d22eb52641489cae18edf992f5af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:10.611113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.611770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46d1f771-4f0f-4cbd-b304-75ee4f63bb3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:10.613338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 977/1972] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:10 2022] DELETE /identity/v3/projects/8e43d22eb52641489cae18edf992f5af => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:10.618179 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e41a7-a1d8-490d-bf38-793a5ef4be64 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.618466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e41a7-a1d8-490d-bf38-793a5ef4be64 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.618765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e41a7-a1d8-490d-bf38-793a5ef4be64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.622202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:10.633553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc673a4-4fa0-4d6f-8ec2-2d9490723c36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:10.634483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc673a4-4fa0-4d6f-8ec2-2d9490723c36 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.635145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc673a4-4fa0-4d6f-8ec2-2d9490723c36 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.635492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc673a4-4fa0-4d6f-8ec2-2d9490723c36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.637428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7cc673a4-4fa0-4d6f-8ec2-2d9490723c36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:10.637917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-467e41a7-a1d8-490d-bf38-793a5ef4be64 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:10.639148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-467e41a7-a1d8-490d-bf38-793a5ef4be64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:10.702766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc673a4-4fa0-4d6f-8ec2-2d9490723c36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:10.704212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc673a4-4fa0-4d6f-8ec2-2d9490723c36 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:10.706781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 978/1973] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:10.714636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516aa55-b0d2-4b80-8e65-077815463b87 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.714962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516aa55-b0d2-4b80-8e65-077815463b87 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.715387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516aa55-b0d2-4b80-8e65-077815463b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.716420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 996/1974] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:10.736106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-4516aa55-b0d2-4b80-8e65-077815463b87 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:10.737782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4516aa55-b0d2-4b80-8e65-077815463b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:10.745730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 979/1975] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:10.750519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc673a4-4fa0-4d6f-8ec2-2d9490723c36 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:10.762178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc673a4-4fa0-4d6f-8ec2-2d9490723c36 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:10.767085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f097-1ffd-44a3-9f43-1c8cf90e1875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:10.767754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f097-1ffd-44a3-9f43-1c8cf90e1875 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.768035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f097-1ffd-44a3-9f43-1c8cf90e1875 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.768314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f097-1ffd-44a3-9f43-1c8cf90e1875 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.769039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f097-1ffd-44a3-9f43-1c8cf90e1875 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:10.769345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f097-1ffd-44a3-9f43-1c8cf90e1875 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:10.770950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f097-1ffd-44a3-9f43-1c8cf90e1875 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:10.783210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 997/1976] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:10.788071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516aa55-b0d2-4b80-8e65-077815463b87 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:10.797594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516aa55-b0d2-4b80-8e65-077815463b87 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:10.802239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3c1a9-6a62-4bc7-9bfd-dc33188debd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:10.802944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3c1a9-6a62-4bc7-9bfd-dc33188debd7 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.803225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3c1a9-6a62-4bc7-9bfd-dc33188debd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.803496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3c1a9-6a62-4bc7-9bfd-dc33188debd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.804171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3c1a9-6a62-4bc7-9bfd-dc33188debd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:10.806120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3c1a9-6a62-4bc7-9bfd-dc33188debd7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:10.832452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 980/1977] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:10 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:10.836644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f097-1ffd-44a3-9f43-1c8cf90e1875 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:10.847820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f097-1ffd-44a3-9f43-1c8cf90e1875 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:10.911992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90d44e-c3a7-4e02-9620-4a8cbf33f38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:10.912668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90d44e-c3a7-4e02-9620-4a8cbf33f38c admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.912949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90d44e-c3a7-4e02-9620-4a8cbf33f38c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.913227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90d44e-c3a7-4e02-9620-4a8cbf33f38c admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.913917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90d44e-c3a7-4e02-9620-4a8cbf33f38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:10.915878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90d44e-c3a7-4e02-9620-4a8cbf33f38c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:10.940508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 998/1978] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:10 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:10.941801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3c1a9-6a62-4bc7-9bfd-dc33188debd7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:10.952792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3c1a9-6a62-4bc7-9bfd-dc33188debd7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:10.957851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c31699-7f74-44da-bdaa-2d1221eae664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:10.958874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c31699-7f74-44da-bdaa-2d1221eae664 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:10.958986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c31699-7f74-44da-bdaa-2d1221eae664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:10.959355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c31699-7f74-44da-bdaa-2d1221eae664 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:10.960064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c31699-7f74-44da-bdaa-2d1221eae664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:10.961829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c31699-7f74-44da-bdaa-2d1221eae664 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:10.979460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 981/1979] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:10.984445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90d44e-c3a7-4e02-9620-4a8cbf33f38c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:10.996788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90d44e-c3a7-4e02-9620-4a8cbf33f38c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.007953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37206ec9-3dfc-49b5-8701-78db8daaa65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.008654 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37206ec9-3dfc-49b5-8701-78db8daaa65e admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.008942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37206ec9-3dfc-49b5-8701-78db8daaa65e admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.009224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37206ec9-3dfc-49b5-8701-78db8daaa65e admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.009963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37206ec9-3dfc-49b5-8701-78db8daaa65e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.011924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37206ec9-3dfc-49b5-8701-78db8daaa65e admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.022819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 999/1980] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:11.027022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c31699-7f74-44da-bdaa-2d1221eae664 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.038106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c31699-7f74-44da-bdaa-2d1221eae664 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.042821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb9bcf-f901-40a3-a159-bd2ff54d01a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.043706 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb9bcf-f901-40a3-a159-bd2ff54d01a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.044090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb9bcf-f901-40a3-a159-bd2ff54d01a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.044400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb9bcf-f901-40a3-a159-bd2ff54d01a1 admin admin] PATH_INFO: `/v3/projects/e8716a5e56ad4fbf864aa44b7932cf50/users/3ac2d23cba384a5cab550ebae9bf525b/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.060627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb9bcf-f901-40a3-a159-bd2ff54d01a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8716a5e56ad4fbf864aa44b7932cf50, user_id=3ac2d23cba384a5cab550ebae9bf525b, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.063393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb9bcf-f901-40a3-a159-bd2ff54d01a1 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.076290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 982/1981] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:11 2022] PUT /identity/v3/projects/e8716a5e56ad4fbf864aa44b7932cf50/users/3ac2d23cba384a5cab550ebae9bf525b/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:11.088165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d09de8-8ae4-483c-ae02-d3b0edabf0e7 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.088592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d09de8-8ae4-483c-ae02-d3b0edabf0e7 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.088920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d09de8-8ae4-483c-ae02-d3b0edabf0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.116824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b1d09de8-8ae4-483c-ae02-d3b0edabf0e7 None None] MFA Rules not processed for user `3ac2d23cba384a5cab550ebae9bf525b`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:11.118066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b1d09de8-8ae4-483c-ae02-d3b0edabf0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:11.164834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:11.183322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1000/1982] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:11 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:11.188352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb9bcf-f901-40a3-a159-bd2ff54d01a1 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.231366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:11.251094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb9bcf-f901-40a3-a159-bd2ff54d01a1 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.255693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385c1a0-8d7d-41eb-91ec-833b285afd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.256346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385c1a0-8d7d-41eb-91ec-833b285afd96 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.256625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385c1a0-8d7d-41eb-91ec-833b285afd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.256898 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385c1a0-8d7d-41eb-91ec-833b285afd96 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.257583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385c1a0-8d7d-41eb-91ec-833b285afd96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.259529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385c1a0-8d7d-41eb-91ec-833b285afd96 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.288284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 983/1983] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:11 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:11.292472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d09de8-8ae4-483c-ae02-d3b0edabf0e7 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.304986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d09de8-8ae4-483c-ae02-d3b0edabf0e7 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.346858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385c1a0-8d7d-41eb-91ec-833b285afd96 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.349276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c385c1a0-8d7d-41eb-91ec-833b285afd96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:11.365870 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9aab2-1dfd-4079-ba99-ddf9aaa9ab92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.366596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9aab2-1dfd-4079-ba99-ddf9aaa9ab92 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.366987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9aab2-1dfd-4079-ba99-ddf9aaa9ab92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.367445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9aab2-1dfd-4079-ba99-ddf9aaa9ab92 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.368210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9aab2-1dfd-4079-ba99-ddf9aaa9ab92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.371517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9aab2-1dfd-4079-ba99-ddf9aaa9ab92 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.396161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1001/1984] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:11 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:11.400243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9aab2-1dfd-4079-ba99-ddf9aaa9ab92 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.410390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9aab2-1dfd-4079-ba99-ddf9aaa9ab92 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.415476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374ccb-d246-4cf5-ac1f-84eba97cfd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.416307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374ccb-d246-4cf5-ac1f-84eba97cfd74 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.416866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374ccb-d246-4cf5-ac1f-84eba97cfd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.417310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374ccb-d246-4cf5-ac1f-84eba97cfd74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.418186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374ccb-d246-4cf5-ac1f-84eba97cfd74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.419998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374ccb-d246-4cf5-ac1f-84eba97cfd74 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.422006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385c1a0-8d7d-41eb-91ec-833b285afd96 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.426841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.427715 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.428003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.428281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] PATH_INFO: `/v3/users/745389010cfa48e18e01489a9b664e95` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.429488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=745389010cfa48e18e01489a9b664e95)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.430382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1002/1985] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:11.434649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374ccb-d246-4cf5-ac1f-84eba97cfd74 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.434922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.444438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374ccb-d246-4cf5-ac1f-84eba97cfd74 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.450235 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a2426-5a13-4195-8a8f-229ffc036ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.451145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a2426-5a13-4195-8a8f-229ffc036ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.451449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a2426-5a13-4195-8a8f-229ffc036ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.452926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a2426-5a13-4195-8a8f-229ffc036ac0 admin admin] PATH_INFO: `/v3/projects/0aa31bf0a6fb4601a08d93ade0d4d9f8/users/0b40d08d72eb4975b763d4322fc883b7/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.470978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a2426-5a13-4195-8a8f-229ffc036ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa31bf0a6fb4601a08d93ade0d4d9f8, user_id=0b40d08d72eb4975b763d4322fc883b7, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.474150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a2426-5a13-4195-8a8f-229ffc036ac0 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.485611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '745389010cfa48e18e01489a9b664e95'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:11.487246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1003/1986] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:11 2022] PUT /identity/v3/projects/0aa31bf0a6fb4601a08d93ade0d4d9f8/users/0b40d08d72eb4975b763d4322fc883b7/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:11.491140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a2426-5a13-4195-8a8f-229ffc036ac0 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.493239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '745389010cfa48e18e01489a9b664e95'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:11.500441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '745389010cfa48e18e01489a9b664e95'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:11.501093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '745389010cfa48e18e01489a9b664e95'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:11.501778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f4a2dee1-2699-4d7b-9a85-ae77b4a7a0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '745389010cfa48e18e01489a9b664e95'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:11.519420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 984/1987] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:11 2022] DELETE /identity/v3/users/745389010cfa48e18e01489a9b664e95 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:11.523595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5750a09-4183-44f5-b53d-cbec1e3551cb None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.523900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5750a09-4183-44f5-b53d-cbec1e3551cb None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.524181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5750a09-4183-44f5-b53d-cbec1e3551cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.542037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d5750a09-4183-44f5-b53d-cbec1e3551cb None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:11.543378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d5750a09-4183-44f5-b53d-cbec1e3551cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:11.554915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a2426-5a13-4195-8a8f-229ffc036ac0 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.583457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:11.605935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:11.607743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 985/1988] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:11.617200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5750a09-4183-44f5-b53d-cbec1e3551cb None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.618882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d5750a09-4183-44f5-b53d-cbec1e3551cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:11.623827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba606e-3e56-4716-a49b-ced7cb474c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.625012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba606e-3e56-4716-a49b-ced7cb474c23 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.625328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba606e-3e56-4716-a49b-ced7cb474c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.625608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba606e-3e56-4716-a49b-ced7cb474c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.626330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba606e-3e56-4716-a49b-ced7cb474c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.628091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba606e-3e56-4716-a49b-ced7cb474c23 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.638920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1004/1989] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:11.644341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba606e-3e56-4716-a49b-ced7cb474c23 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.653141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-76ba606e-3e56-4716-a49b-ced7cb474c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:11.659772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba606e-3e56-4716-a49b-ced7cb474c23 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.664670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7894a93-cf88-4fda-a85e-250082172143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.665543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7894a93-cf88-4fda-a85e-250082172143 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.665820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7894a93-cf88-4fda-a85e-250082172143 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.666093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7894a93-cf88-4fda-a85e-250082172143 admin admin] PATH_INFO: `/v3/projects/0aa31bf0a6fb4601a08d93ade0d4d9f8/users/0b40d08d72eb4975b763d4322fc883b7/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.668086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7894a93-cf88-4fda-a85e-250082172143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa31bf0a6fb4601a08d93ade0d4d9f8, user_id=0b40d08d72eb4975b763d4322fc883b7, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.670688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7894a93-cf88-4fda-a85e-250082172143 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.680074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5750a09-4183-44f5-b53d-cbec1e3551cb None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.680873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1005/1990] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:11 2022] PUT /identity/v3/projects/0aa31bf0a6fb4601a08d93ade0d4d9f8/users/0b40d08d72eb4975b763d4322fc883b7/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:11.691724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce69de6-1c48-4375-af48-d45d9c457aa2 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.692017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce69de6-1c48-4375-af48-d45d9c457aa2 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.692305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce69de6-1c48-4375-af48-d45d9c457aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.719356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-0ce69de6-1c48-4375-af48-d45d9c457aa2 None None] MFA Rules not processed for user `0b40d08d72eb4975b763d4322fc883b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:11.720569 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0ce69de6-1c48-4375-af48-d45d9c457aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:11.737153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91f25f-0889-4ade-b310-7b66dc179308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.737805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91f25f-0889-4ade-b310-7b66dc179308 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.738092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91f25f-0889-4ade-b310-7b66dc179308 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.738377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91f25f-0889-4ade-b310-7b66dc179308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.790438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1006/1991] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:11.801724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce69de6-1c48-4375-af48-d45d9c457aa2 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.803116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0ce69de6-1c48-4375-af48-d45d9c457aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:11.807259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91f25f-0889-4ade-b310-7b66dc179308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.808682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91f25f-0889-4ade-b310-7b66dc179308 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.818864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 986/1992] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:11.861906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce69de6-1c48-4375-af48-d45d9c457aa2 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.865805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8db6ca-abc9-4a7d-9fe8-cbfb0d9450d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.866466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8db6ca-abc9-4a7d-9fe8-cbfb0d9450d7 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.866771 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8db6ca-abc9-4a7d-9fe8-cbfb0d9450d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.867058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8db6ca-abc9-4a7d-9fe8-cbfb0d9450d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.878277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8db6ca-abc9-4a7d-9fe8-cbfb0d9450d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:11.879988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8db6ca-abc9-4a7d-9fe8-cbfb0d9450d7 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:11.889848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1007/1993] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:11.963096 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91f25f-0889-4ade-b310-7b66dc179308 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:11.972803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91f25f-0889-4ade-b310-7b66dc179308 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:11.976501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731bc31-8189-4c3c-bd6c-a04fb6d0bba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:11.977143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731bc31-8189-4c3c-bd6c-a04fb6d0bba2 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:11.977456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731bc31-8189-4c3c-bd6c-a04fb6d0bba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:11.977760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731bc31-8189-4c3c-bd6c-a04fb6d0bba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:11.979552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2731bc31-8189-4c3c-bd6c-a04fb6d0bba2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:12.034790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:12.049313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731bc31-8189-4c3c-bd6c-a04fb6d0bba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:12.050801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731bc31-8189-4c3c-bd6c-a04fb6d0bba2 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:12.061229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 987/1994] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:11 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:12.118255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8db6ca-abc9-4a7d-9fe8-cbfb0d9450d7 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:12.120700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1b8db6ca-abc9-4a7d-9fe8-cbfb0d9450d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:12.143887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8db6ca-abc9-4a7d-9fe8-cbfb0d9450d7 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:12.149178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:12.150222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:12.150529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:12.150936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] PATH_INFO: `/v3/projects/452f47b13af24caf979895f41f240e48` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:12.152371 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=452f47b13af24caf979895f41f240e48)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:12.154233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:12.217475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '452f47b13af24caf979895f41f240e48', 'request_id': 'req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.223704 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '452f47b13af24caf979895f41f240e48', 'request_id': 'req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.227355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:12.231987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '452f47b13af24caf979895f41f240e48', 'request_id': 'req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.244938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] The token cache is being invalidate because project 452f47b13af24caf979895f41f240e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:12.245227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.245895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.247630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1008/1995] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:12 2022] DELETE /identity/v3/projects/452f47b13af24caf979895f41f240e48 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:12.252540 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731bc31-8189-4c3c-bd6c-a04fb6d0bba2 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:12.263379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731bc31-8189-4c3c-bd6c-a04fb6d0bba2 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:12.319949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:12.320742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:12.321030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:12.321308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] PATH_INFO: `/v3/users/34be39bf3bab4e699f660c0e8523422b` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:12.322484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34be39bf3bab4e699f660c0e8523422b)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:12.324466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:12.369655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34be39bf3bab4e699f660c0e8523422b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.377274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34be39bf3bab4e699f660c0e8523422b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.384212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34be39bf3bab4e699f660c0e8523422b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.384846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34be39bf3bab4e699f660c0e8523422b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.385461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34be39bf3bab4e699f660c0e8523422b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.406292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 988/1996] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:12 2022] DELETE /identity/v3/users/34be39bf3bab4e699f660c0e8523422b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:12.679471 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:12.681054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:12.695748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff0524-a5e8-4fa5-9c3b-6635f3587075 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:12.754700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:12.755598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:12.755877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:12.756163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] PATH_INFO: `/v3/projects/2f02aa8e85f74e58a9a2642158bbe7de` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:12.757332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f02aa8e85f74e58a9a2642158bbe7de)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:12.759266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:12.807995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f02aa8e85f74e58a9a2642158bbe7de', 'request_id': 'req-da6d8274-c962-46cf-8631-a55c75a4e513'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.814994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f02aa8e85f74e58a9a2642158bbe7de', 'request_id': 'req-da6d8274-c962-46cf-8631-a55c75a4e513'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.827148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f02aa8e85f74e58a9a2642158bbe7de', 'request_id': 'req-da6d8274-c962-46cf-8631-a55c75a4e513'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.838484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] The token cache is being invalidate because project 2f02aa8e85f74e58a9a2642158bbe7de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:12.838819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.839451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.840996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1009/1997] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:12 2022] DELETE /identity/v3/projects/2f02aa8e85f74e58a9a2642158bbe7de => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:12.845609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:12.847065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:12.859601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a888e6-c8a4-4d5d-ac35-d9d127bdb0b6 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:12.913907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:12.914802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:12.915086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:12.915360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] PATH_INFO: `/v3/users/848f87a819894f7898dbc1b9b196248c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:12.916517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=848f87a819894f7898dbc1b9b196248c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:12.918431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:12.928038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:12.962792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '848f87a819894f7898dbc1b9b196248c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.970426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '848f87a819894f7898dbc1b9b196248c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.977190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '848f87a819894f7898dbc1b9b196248c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.977902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '848f87a819894f7898dbc1b9b196248c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.978419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-677376ac-fec7-4eb6-b69e-80184db8eb9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '848f87a819894f7898dbc1b9b196248c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:12.995243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 989/1998] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:12 2022] DELETE /identity/v3/users/848f87a819894f7898dbc1b9b196248c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:13.284163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:13.289254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:13.316820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d8274-c962-46cf-8631-a55c75a4e513 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:13.416494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:13.417376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.417651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:13.417924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] PATH_INFO: `/v3/projects/c09096cc2ab54c858667511df6db46c4` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:13.419197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09096cc2ab54c858667511df6db46c4)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:13.429466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:13.451351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:13.491194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09096cc2ab54c858667511df6db46c4', 'request_id': 'req-72f6e32d-30bf-45ea-8963-3165b87e5f91'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:13.500159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09096cc2ab54c858667511df6db46c4', 'request_id': 'req-72f6e32d-30bf-45ea-8963-3165b87e5f91'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:13.509423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09096cc2ab54c858667511df6db46c4', 'request_id': 'req-72f6e32d-30bf-45ea-8963-3165b87e5f91'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:13.521919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] The token cache is being invalidate because project c09096cc2ab54c858667511df6db46c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:13.522211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:13.522892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-72f6e32d-30bf-45ea-8963-3165b87e5f91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:13.524642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1010/1999] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:13 2022] DELETE /identity/v3/projects/c09096cc2ab54c858667511df6db46c4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:13.531046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7524a9-8720-450a-b225-f33332bcbe3e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.531350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7524a9-8720-450a-b225-f33332bcbe3e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:13.531839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7524a9-8720-450a-b225-f33332bcbe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:13.551763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-cc7524a9-8720-450a-b225-f33332bcbe3e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:13.553201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cc7524a9-8720-450a-b225-f33332bcbe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:13.643164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 990/2000] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:13.651502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e57bf-47a8-4619-8e00-5664911d9040 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.653363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e57bf-47a8-4619-8e00-5664911d9040 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:13.653363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e57bf-47a8-4619-8e00-5664911d9040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:13.674300 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6c4e57bf-47a8-4619-8e00-5664911d9040 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:13.675840 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6c4e57bf-47a8-4619-8e00-5664911d9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:13.685426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1011/2001] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:13.690253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7524a9-8720-450a-b225-f33332bcbe3e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:13.699928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7524a9-8720-450a-b225-f33332bcbe3e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:13.704519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc39f1-e2c1-4491-b1e9-c0e95b2d3c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:13.705238 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc39f1-e2c1-4491-b1e9-c0e95b2d3c3b admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.705528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc39f1-e2c1-4491-b1e9-c0e95b2d3c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:13.705809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc39f1-e2c1-4491-b1e9-c0e95b2d3c3b admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:13.706534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc39f1-e2c1-4491-b1e9-c0e95b2d3c3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:13.706989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc39f1-e2c1-4491-b1e9-c0e95b2d3c3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:13.708588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc39f1-e2c1-4491-b1e9-c0e95b2d3c3b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:13.720455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 991/2002] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:13.724766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e57bf-47a8-4619-8e00-5664911d9040 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:13.734625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e57bf-47a8-4619-8e00-5664911d9040 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:13.740195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e79538-18cd-43e7-adad-582019e96f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:13.740860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e79538-18cd-43e7-adad-582019e96f89 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.741143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e79538-18cd-43e7-adad-582019e96f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:13.741427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e79538-18cd-43e7-adad-582019e96f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:13.742115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e79538-18cd-43e7-adad-582019e96f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:13.744115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e79538-18cd-43e7-adad-582019e96f89 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:13.774401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1012/2003] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:13 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:13.775518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc39f1-e2c1-4491-b1e9-c0e95b2d3c3b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:13.785496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc39f1-e2c1-4491-b1e9-c0e95b2d3c3b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:13.841600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228ae20-bdd2-4137-a844-22b62e1d9fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:13.842267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228ae20-bdd2-4137-a844-22b62e1d9fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.842553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228ae20-bdd2-4137-a844-22b62e1d9fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:13.842864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228ae20-bdd2-4137-a844-22b62e1d9fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:13.844086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228ae20-bdd2-4137-a844-22b62e1d9fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:13.845988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228ae20-bdd2-4137-a844-22b62e1d9fe9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:13.867700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 992/2004] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:13 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:13.871603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e79538-18cd-43e7-adad-582019e96f89 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:13.881908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e79538-18cd-43e7-adad-582019e96f89 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:13.886396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557402a5-f25f-4e7a-aefc-18e663ba7583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:13.887087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557402a5-f25f-4e7a-aefc-18e663ba7583 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.887365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557402a5-f25f-4e7a-aefc-18e663ba7583 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:13.887636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557402a5-f25f-4e7a-aefc-18e663ba7583 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:13.888391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557402a5-f25f-4e7a-aefc-18e663ba7583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:13.889992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557402a5-f25f-4e7a-aefc-18e663ba7583 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:13.900225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1013/2005] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:13.904298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228ae20-bdd2-4137-a844-22b62e1d9fe9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:13.911393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:13.915592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228ae20-bdd2-4137-a844-22b62e1d9fe9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:13.922374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d8de0-b40e-4b02-a787-db2481ea40b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:13.923300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d8de0-b40e-4b02-a787-db2481ea40b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.923602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d8de0-b40e-4b02-a787-db2481ea40b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:13.923928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d8de0-b40e-4b02-a787-db2481ea40b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:13.924862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d8de0-b40e-4b02-a787-db2481ea40b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:13.926477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d8de0-b40e-4b02-a787-db2481ea40b9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:13.936818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 993/2006] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:13.940872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557402a5-f25f-4e7a-aefc-18e663ba7583 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:13.950546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557402a5-f25f-4e7a-aefc-18e663ba7583 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:13.955330 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e5cac-d9af-429b-8bd2-10c9a31c7c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:13.956205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e5cac-d9af-429b-8bd2-10c9a31c7c3b admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.956483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e5cac-d9af-429b-8bd2-10c9a31c7c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:13.956761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e5cac-d9af-429b-8bd2-10c9a31c7c3b admin admin] PATH_INFO: `/v3/projects/f025a34ae83e458585973f05e50c5809/users/6989fa1638584b74994c526b2f7241b4/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:13.972106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e5cac-d9af-429b-8bd2-10c9a31c7c3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f025a34ae83e458585973f05e50c5809, user_id=6989fa1638584b74994c526b2f7241b4, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:13.974816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e5cac-d9af-429b-8bd2-10c9a31c7c3b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:13.988191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1014/2007] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:13 2022] PUT /identity/v3/projects/f025a34ae83e458585973f05e50c5809/users/6989fa1638584b74994c526b2f7241b4/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:13.999367 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd2464-1913-411a-b822-ae67e8e78da4 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:13.999740 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd2464-1913-411a-b822-ae67e8e78da4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:14.000036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd2464-1913-411a-b822-ae67e8e78da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:14.025582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9ffd2464-1913-411a-b822-ae67e8e78da4 None None] MFA Rules not processed for user `6989fa1638584b74994c526b2f7241b4`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:14.026807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9ffd2464-1913-411a-b822-ae67e8e78da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:14.089502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 994/2008] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:14.101077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e5cac-d9af-429b-8bd2-10c9a31c7c3b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:14.102797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-573e5cac-d9af-429b-8bd2-10c9a31c7c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:14.158634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e5cac-d9af-429b-8bd2-10c9a31c7c3b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:14.162495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27c6eb-de78-4bd8-87c4-353d46d5cf02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:14.163163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27c6eb-de78-4bd8-87c4-353d46d5cf02 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:14.163443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27c6eb-de78-4bd8-87c4-353d46d5cf02 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:14.163716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27c6eb-de78-4bd8-87c4-353d46d5cf02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:14.172416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27c6eb-de78-4bd8-87c4-353d46d5cf02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:14.173827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27c6eb-de78-4bd8-87c4-353d46d5cf02 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:14.182685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1015/2009] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:17.064716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd2464-1913-411a-b822-ae67e8e78da4 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:17.066215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9ffd2464-1913-411a-b822-ae67e8e78da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:17.132520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd2464-1913-411a-b822-ae67e8e78da4 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:17.137040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:17.137908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.138200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.138481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] PATH_INFO: `/v3/users/fbfba31fa3834b559828f527ab7f1a9a` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.139688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbfba31fa3834b559828f527ab7f1a9a)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:17.142178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:17.187811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbfba31fa3834b559828f527ab7f1a9a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.196112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbfba31fa3834b559828f527ab7f1a9a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.204560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbfba31fa3834b559828f527ab7f1a9a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.205296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbfba31fa3834b559828f527ab7f1a9a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.205919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbfba31fa3834b559828f527ab7f1a9a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.222823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 995/2010] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:17 2022] DELETE /identity/v3/users/fbfba31fa3834b559828f527ab7f1a9a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:17.227303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eb39d-e97d-486f-bd5b-e843dfc527c1 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.227777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eb39d-e97d-486f-bd5b-e843dfc527c1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.227871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eb39d-e97d-486f-bd5b-e843dfc527c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.245259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-713eb39d-e97d-486f-bd5b-e843dfc527c1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:17.246653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-713eb39d-e97d-486f-bd5b-e843dfc527c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:17.289533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:17.309565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1016/2011] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:17.319896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:17.321314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:17.326972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:17.387562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2265c-0a64-409f-802a-d809dbbf31a9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:17.391199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9e033-e66e-41ae-93f5-8262da60e691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:17.391856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9e033-e66e-41ae-93f5-8262da60e691 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.392144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9e033-e66e-41ae-93f5-8262da60e691 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.392419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9e033-e66e-41ae-93f5-8262da60e691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.402986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9e033-e66e-41ae-93f5-8262da60e691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:17.404419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9e033-e66e-41ae-93f5-8262da60e691 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:17.414747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 996/2012] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:17.638923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eb39d-e97d-486f-bd5b-e843dfc527c1 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:17.640329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-713eb39d-e97d-486f-bd5b-e843dfc527c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:17.652198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eb39d-e97d-486f-bd5b-e843dfc527c1 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:17.657361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:17.658228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.658506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.658815 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] PATH_INFO: `/v3/users/1e513343b44f408ca7d16e019c8a02ce` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.659996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e513343b44f408ca7d16e019c8a02ce)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:17.662164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:17.704069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9e033-e66e-41ae-93f5-8262da60e691 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:17.705894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cbe9e033-e66e-41ae-93f5-8262da60e691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:17.719679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e513343b44f408ca7d16e019c8a02ce'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.727634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e513343b44f408ca7d16e019c8a02ce'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.729962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9e033-e66e-41ae-93f5-8262da60e691 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:17.735939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:17.736342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e513343b44f408ca7d16e019c8a02ce'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.736801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.737036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.737148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e513343b44f408ca7d16e019c8a02ce'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.737314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] PATH_INFO: `/v3/projects/b3e7025f89b941c18111abaedd708554` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.738475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e7025f89b941c18111abaedd708554)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:17.740422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:17.749706 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-94897a55-f463-43e3-a3d5-d551d0f6f3f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e513343b44f408ca7d16e019c8a02ce'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.756336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1017/2013] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:17 2022] DELETE /identity/v3/users/1e513343b44f408ca7d16e019c8a02ce => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:17.760567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331a8a2-8f11-416e-ae20-0d59d79ed2cb None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.761001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331a8a2-8f11-416e-ae20-0d59d79ed2cb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.761352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331a8a2-8f11-416e-ae20-0d59d79ed2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.780041 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4331a8a2-8f11-416e-ae20-0d59d79ed2cb None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:17.781773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4331a8a2-8f11-416e-ae20-0d59d79ed2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:17.791376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e7025f89b941c18111abaedd708554', 'request_id': 'req-d81340d5-257b-4b98-96a5-bbda06444ccd'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.798036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e7025f89b941c18111abaedd708554', 'request_id': 'req-d81340d5-257b-4b98-96a5-bbda06444ccd'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.805339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e7025f89b941c18111abaedd708554', 'request_id': 'req-d81340d5-257b-4b98-96a5-bbda06444ccd'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.816532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] The token cache is being invalidate because project b3e7025f89b941c18111abaedd708554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:17.816841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.817512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d81340d5-257b-4b98-96a5-bbda06444ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:17.819225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 997/2014] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:17 2022] DELETE /identity/v3/projects/b3e7025f89b941c18111abaedd708554 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:17.824002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10581989-1cc3-48a9-bf44-d4ab1d55b8a0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.824471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10581989-1cc3-48a9-bf44-d4ab1d55b8a0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.824764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10581989-1cc3-48a9-bf44-d4ab1d55b8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.841741 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1018/2015] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:17.842090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-10581989-1cc3-48a9-bf44-d4ab1d55b8a0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:17.843857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-10581989-1cc3-48a9-bf44-d4ab1d55b8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:17.851823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 998/2016] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:17.852400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331a8a2-8f11-416e-ae20-0d59d79ed2cb None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:17.853722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4331a8a2-8f11-416e-ae20-0d59d79ed2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:17.859755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d689f79-e15e-4475-a0fa-758d37a8da93 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.860051 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d689f79-e15e-4475-a0fa-758d37a8da93 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.860336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d689f79-e15e-4475-a0fa-758d37a8da93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.877889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-6d689f79-e15e-4475-a0fa-758d37a8da93 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:17.879114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6d689f79-e15e-4475-a0fa-758d37a8da93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:17.887145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 999/2017] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:17.891357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d689f79-e15e-4475-a0fa-758d37a8da93 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:17.900743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d689f79-e15e-4475-a0fa-758d37a8da93 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:17.906786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976a822-fb29-4b79-9396-4b35e5ed0bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:17.907108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976a822-fb29-4b79-9396-4b35e5ed0bfe admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.907388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976a822-fb29-4b79-9396-4b35e5ed0bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.907663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976a822-fb29-4b79-9396-4b35e5ed0bfe admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.908393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976a822-fb29-4b79-9396-4b35e5ed0bfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:17.908745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976a822-fb29-4b79-9396-4b35e5ed0bfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:17.910937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976a822-fb29-4b79-9396-4b35e5ed0bfe admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:17.914186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331a8a2-8f11-416e-ae20-0d59d79ed2cb None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:17.918258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab15ac-f2c5-403d-a1f1-4dca8908fd92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:17.918946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab15ac-f2c5-403d-a1f1-4dca8908fd92 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.919232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab15ac-f2c5-403d-a1f1-4dca8908fd92 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.919534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab15ac-f2c5-403d-a1f1-4dca8908fd92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.924838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1000/2018] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:51:17.928939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976a822-fb29-4b79-9396-4b35e5ed0bfe admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:17.929885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab15ac-f2c5-403d-a1f1-4dca8908fd92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:17.931380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab15ac-f2c5-403d-a1f1-4dca8908fd92 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:17.938589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976a822-fb29-4b79-9396-4b35e5ed0bfe admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:17.941065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1019/2019] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:17.943535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2ee81-6f28-4403-a988-798c0dbc328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:17.944264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2ee81-6f28-4403-a988-798c0dbc328c admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:17.944632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2ee81-6f28-4403-a988-798c0dbc328c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:17.944984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2ee81-6f28-4403-a988-798c0dbc328c admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:17.945748 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2ee81-6f28-4403-a988-798c0dbc328c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:17.947745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2ee81-6f28-4403-a988-798c0dbc328c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:17.976800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1001/2020] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:17 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:17.981267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab15ac-f2c5-403d-a1f1-4dca8908fd92 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:17.986582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:17.993915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab15ac-f2c5-403d-a1f1-4dca8908fd92 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.064318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538cfc8-f1a6-48b9-9eda-63417115beca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.064986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538cfc8-f1a6-48b9-9eda-63417115beca admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.065270 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538cfc8-f1a6-48b9-9eda-63417115beca admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.065539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538cfc8-f1a6-48b9-9eda-63417115beca admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.066240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538cfc8-f1a6-48b9-9eda-63417115beca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.068203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538cfc8-f1a6-48b9-9eda-63417115beca admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.093279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1020/2021] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:17 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:18.095358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2ee81-6f28-4403-a988-798c0dbc328c admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.106320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2ee81-6f28-4403-a988-798c0dbc328c admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.111871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb13566-0641-40b3-8fcd-1849db55d54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.112546 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb13566-0641-40b3-8fcd-1849db55d54b admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.112834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb13566-0641-40b3-8fcd-1849db55d54b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.113466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb13566-0641-40b3-8fcd-1849db55d54b admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.113842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb13566-0641-40b3-8fcd-1849db55d54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.115509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb13566-0641-40b3-8fcd-1849db55d54b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.125928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1002/2022] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:18.129941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538cfc8-f1a6-48b9-9eda-63417115beca admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.140433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538cfc8-f1a6-48b9-9eda-63417115beca admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.145090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912e3a3-66f7-4cde-aa82-6214dea38a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.145687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912e3a3-66f7-4cde-aa82-6214dea38a99 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.145964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912e3a3-66f7-4cde-aa82-6214dea38a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.146238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912e3a3-66f7-4cde-aa82-6214dea38a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.147014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912e3a3-66f7-4cde-aa82-6214dea38a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.148608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912e3a3-66f7-4cde-aa82-6214dea38a99 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.159467 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1021/2023] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:18.163718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb13566-0641-40b3-8fcd-1849db55d54b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.173933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb13566-0641-40b3-8fcd-1849db55d54b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.179342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed82332-405e-438f-b95a-1786010e5e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.180274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed82332-405e-438f-b95a-1786010e5e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.180551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed82332-405e-438f-b95a-1786010e5e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.180825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed82332-405e-438f-b95a-1786010e5e85 admin admin] PATH_INFO: `/v3/projects/975da5681ab94b11b331cf7834bae144/users/fe3e0fe92cf544aa80e4e87877706f88/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.196429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed82332-405e-438f-b95a-1786010e5e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=975da5681ab94b11b331cf7834bae144, user_id=fe3e0fe92cf544aa80e4e87877706f88, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.199136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed82332-405e-438f-b95a-1786010e5e85 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.216210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1003/2024] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:18 2022] PUT /identity/v3/projects/975da5681ab94b11b331cf7834bae144/users/fe3e0fe92cf544aa80e4e87877706f88/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:18.228075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aead3-7fa1-48d8-97c4-fac77fa75d96 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.228534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aead3-7fa1-48d8-97c4-fac77fa75d96 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.231428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aead3-7fa1-48d8-97c4-fac77fa75d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.232403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed82332-405e-438f-b95a-1786010e5e85 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.234197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-eed82332-405e-438f-b95a-1786010e5e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:18.241698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:18.260614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2f0aead3-7fa1-48d8-97c4-fac77fa75d96 None None] MFA Rules not processed for user `fe3e0fe92cf544aa80e4e87877706f88`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:18.262085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2f0aead3-7fa1-48d8-97c4-fac77fa75d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:18.305462 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed82332-405e-438f-b95a-1786010e5e85 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.310198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.311141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.311423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.312237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] PATH_INFO: `/v3/projects/764a223e4ade426ea0ee2c72e6907f63` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.313396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=764a223e4ade426ea0ee2c72e6907f63)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.315410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.333416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1022/2025] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:18.337964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0aead3-7fa1-48d8-97c4-fac77fa75d96 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.349367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0aead3-7fa1-48d8-97c4-fac77fa75d96 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.363060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764a223e4ade426ea0ee2c72e6907f63', 'request_id': 'req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.369375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764a223e4ade426ea0ee2c72e6907f63', 'request_id': 'req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.377284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764a223e4ade426ea0ee2c72e6907f63', 'request_id': 'req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.388033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] The token cache is being invalidate because project 764a223e4ade426ea0ee2c72e6907f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:18.388401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.389112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.391453 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1004/2026] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:18 2022] DELETE /identity/v3/projects/764a223e4ade426ea0ee2c72e6907f63 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:18.395303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.396571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:18.408647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffdfe8-3a21-4ffe-8d51-337be10262b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.409394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffdfe8-3a21-4ffe-8d51-337be10262b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.409679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffdfe8-3a21-4ffe-8d51-337be10262b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.409958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffdfe8-3a21-4ffe-8d51-337be10262b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.410647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffdfe8-3a21-4ffe-8d51-337be10262b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.412858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffdfe8-3a21-4ffe-8d51-337be10262b1 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.426362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:18.440964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1023/2027] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:18 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:18.445008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffdfe8-3a21-4ffe-8d51-337be10262b1 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.446922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-11ffdfe8-3a21-4ffe-8d51-337be10262b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:18.468111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6a801-f2ca-42e1-a59b-e7b58f4b91b5 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.520361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffdfe8-3a21-4ffe-8d51-337be10262b1 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.525635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487eb414-15f5-4a08-b2c5-f74b9533291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.527252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487eb414-15f5-4a08-b2c5-f74b9533291b admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.527644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487eb414-15f5-4a08-b2c5-f74b9533291b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.528027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487eb414-15f5-4a08-b2c5-f74b9533291b admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.528830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487eb414-15f5-4a08-b2c5-f74b9533291b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.528946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:18.531858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487eb414-15f5-4a08-b2c5-f74b9533291b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.549906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.550965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.551339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.551720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] PATH_INFO: `/v3/users/67967fd102a64f7e94821f2e86531eca` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.553027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67967fd102a64f7e94821f2e86531eca)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.555094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.558774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1024/2028] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:18 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:18.566638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487eb414-15f5-4a08-b2c5-f74b9533291b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.580049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487eb414-15f5-4a08-b2c5-f74b9533291b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.585534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9d024-9239-4d99-8edc-9d05a54618be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.586308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9d024-9239-4d99-8edc-9d05a54618be admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.586718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9d024-9239-4d99-8edc-9d05a54618be admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.587060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9d024-9239-4d99-8edc-9d05a54618be admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.587814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9d024-9239-4d99-8edc-9d05a54618be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.590277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9d024-9239-4d99-8edc-9d05a54618be admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.601550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1025/2029] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:18.605708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9d024-9239-4d99-8edc-9d05a54618be admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.615273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67967fd102a64f7e94821f2e86531eca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.628067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9d024-9239-4d99-8edc-9d05a54618be admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.631436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67967fd102a64f7e94821f2e86531eca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.636414 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9fdd7-631d-4068-be77-3dc94d041047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.636959 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9fdd7-631d-4068-be77-3dc94d041047 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.637487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9fdd7-631d-4068-be77-3dc94d041047 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.637855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9fdd7-631d-4068-be77-3dc94d041047 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.638679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9fdd7-631d-4068-be77-3dc94d041047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.641190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9fdd7-631d-4068-be77-3dc94d041047 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.644000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67967fd102a64f7e94821f2e86531eca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.645621 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67967fd102a64f7e94821f2e86531eca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.646564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a91dec85-1c65-4aa1-8c19-7fc01368fa88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67967fd102a64f7e94821f2e86531eca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:18.658859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1026/2030] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:18.659915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9fdd7-631d-4068-be77-3dc94d041047 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.661951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-77f9fdd7-631d-4068-be77-3dc94d041047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:18.671509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1005/2031] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:18 2022] DELETE /identity/v3/users/67967fd102a64f7e94821f2e86531eca => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:18.678681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9fdd7-631d-4068-be77-3dc94d041047 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.744125 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2278aa-2011-4e69-be69-7b431b678694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.744988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2278aa-2011-4e69-be69-7b431b678694 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.745302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2278aa-2011-4e69-be69-7b431b678694 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.745585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2278aa-2011-4e69-be69-7b431b678694 admin admin] PATH_INFO: `/v3/projects/51c3d83d26bc486f857ae4ffa7894bd3/users/14d980b05fcd4f7cbbe7f1a3e5551ca4/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.761242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2278aa-2011-4e69-be69-7b431b678694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c3d83d26bc486f857ae4ffa7894bd3, user_id=14d980b05fcd4f7cbbe7f1a3e5551ca4, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.764075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2278aa-2011-4e69-be69-7b431b678694 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:18.778021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1027/2032] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:18 2022] PUT /identity/v3/projects/51c3d83d26bc486f857ae4ffa7894bd3/users/14d980b05fcd4f7cbbe7f1a3e5551ca4/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:18.790189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab22fd-aea6-4b90-ab6c-7a588c71eac7 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.790524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab22fd-aea6-4b90-ab6c-7a588c71eac7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.790850 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab22fd-aea6-4b90-ab6c-7a588c71eac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.822646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-4bab22fd-aea6-4b90-ab6c-7a588c71eac7 None None] MFA Rules not processed for user `14d980b05fcd4f7cbbe7f1a3e5551ca4`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:18.823905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4bab22fd-aea6-4b90-ab6c-7a588c71eac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:18.892234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1006/2033] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:18.896930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2278aa-2011-4e69-be69-7b431b678694 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.945606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab22fd-aea6-4b90-ab6c-7a588c71eac7 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:18.947034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4bab22fd-aea6-4b90-ab6c-7a588c71eac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:18.952724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:18.979212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2278aa-2011-4e69-be69-7b431b678694 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:18.984963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032ee13-94bd-4420-9860-85a28f75d38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:18.985688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032ee13-94bd-4420-9860-85a28f75d38a admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:18.985970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032ee13-94bd-4420-9860-85a28f75d38a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:18.986247 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032ee13-94bd-4420-9860-85a28f75d38a admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:18.987011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032ee13-94bd-4420-9860-85a28f75d38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:18.988954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032ee13-94bd-4420-9860-85a28f75d38a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.012084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:19.016399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1028/2034] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:18 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:19.020593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032ee13-94bd-4420-9860-85a28f75d38a admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.029996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab22fd-aea6-4b90-ab6c-7a588c71eac7 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.094993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.096003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.096541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.096699 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] PATH_INFO: `/v3/projects/e2c6c6ad15f944b6bd7177b99fffa1fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.097964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2c6c6ad15f944b6bd7177b99fffa1fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.098230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032ee13-94bd-4420-9860-85a28f75d38a admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.100176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.103030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505aa32-22d1-4367-9ce9-244e0ac07ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.103699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505aa32-22d1-4367-9ce9-244e0ac07ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.103986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505aa32-22d1-4367-9ce9-244e0ac07ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.104285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505aa32-22d1-4367-9ce9-244e0ac07ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.104951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505aa32-22d1-4367-9ce9-244e0ac07ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.106894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505aa32-22d1-4367-9ce9-244e0ac07ac2 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.128814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1029/2035] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:19 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:19.133165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505aa32-22d1-4367-9ce9-244e0ac07ac2 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.142944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505aa32-22d1-4367-9ce9-244e0ac07ac2 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.149644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2c6c6ad15f944b6bd7177b99fffa1fa', 'request_id': 'req-95d87719-0fbd-4823-9f4e-fdc9933643d3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:19.157309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2c6c6ad15f944b6bd7177b99fffa1fa', 'request_id': 'req-95d87719-0fbd-4823-9f4e-fdc9933643d3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:19.166057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2c6c6ad15f944b6bd7177b99fffa1fa', 'request_id': 'req-95d87719-0fbd-4823-9f4e-fdc9933643d3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:19.178043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] The token cache is being invalidate because project e2c6c6ad15f944b6bd7177b99fffa1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:19.178341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:19.179028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-95d87719-0fbd-4823-9f4e-fdc9933643d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:19.180585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1007/2036] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:18 2022] DELETE /identity/v3/projects/e2c6c6ad15f944b6bd7177b99fffa1fa => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:19.185392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8eae-7b95-4ed8-8af4-baf6dc830f86 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.185681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8eae-7b95-4ed8-8af4-baf6dc830f86 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.185962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8eae-7b95-4ed8-8af4-baf6dc830f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.204271 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-087b8eae-7b95-4ed8-8af4-baf6dc830f86 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:19.205566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-087b8eae-7b95-4ed8-8af4-baf6dc830f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:19.208821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254babfa-5cc9-404f-8575-284a8fa9d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.209496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254babfa-5cc9-404f-8575-284a8fa9d79e admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.209786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254babfa-5cc9-404f-8575-284a8fa9d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.210064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254babfa-5cc9-404f-8575-284a8fa9d79e admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.210820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254babfa-5cc9-404f-8575-284a8fa9d79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.212415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254babfa-5cc9-404f-8575-284a8fa9d79e admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.213597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1008/2037] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:19.221568 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c87b8-5adc-4056-be01-b4af521cf69b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.221868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c87b8-5adc-4056-be01-b4af521cf69b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.222149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c87b8-5adc-4056-be01-b4af521cf69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.226414 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1030/2038] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:19.227206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254babfa-5cc9-404f-8575-284a8fa9d79e admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.228922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-254babfa-5cc9-404f-8575-284a8fa9d79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:19.240901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-378c87b8-5adc-4056-be01-b4af521cf69b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:19.242093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-378c87b8-5adc-4056-be01-b4af521cf69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:19.242208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254babfa-5cc9-404f-8575-284a8fa9d79e admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.246769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c534744-1746-437c-b18d-cac4f700c004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.247665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c534744-1746-437c-b18d-cac4f700c004 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.247898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c534744-1746-437c-b18d-cac4f700c004 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.248166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c534744-1746-437c-b18d-cac4f700c004 admin admin] PATH_INFO: `/v3/projects/e060c3513c8c4c459fe9ef1697b77963/users/5b4a5836fd134ebba97249c2cae943fe/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.250013 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1009/2039] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:19.254364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c87b8-5adc-4056-be01-b4af521cf69b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.262792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c534744-1746-437c-b18d-cac4f700c004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e060c3513c8c4c459fe9ef1697b77963, user_id=5b4a5836fd134ebba97249c2cae943fe, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.263724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c87b8-5adc-4056-be01-b4af521cf69b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.265451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c534744-1746-437c-b18d-cac4f700c004 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.268581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6d24b-da18-4f74-9c75-04866aa8a5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.269224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6d24b-da18-4f74-9c75-04866aa8a5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.269526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6d24b-da18-4f74-9c75-04866aa8a5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.269789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6d24b-da18-4f74-9c75-04866aa8a5d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.270587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6d24b-da18-4f74-9c75-04866aa8a5d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:19.270931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6d24b-da18-4f74-9c75-04866aa8a5d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.272863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6d24b-da18-4f74-9c75-04866aa8a5d9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.284506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1031/2040] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:19 2022] PUT /identity/v3/projects/e060c3513c8c4c459fe9ef1697b77963/users/5b4a5836fd134ebba97249c2cae943fe/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:19.290154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1010/2041] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:51:19.290311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c534744-1746-437c-b18d-cac4f700c004 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.291071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6d24b-da18-4f74-9c75-04866aa8a5d9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.301315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6d24b-da18-4f74-9c75-04866aa8a5d9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.323112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:19.351813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c534744-1746-437c-b18d-cac4f700c004 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.356502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff158c4e-537c-4377-a869-f9da130683f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.357160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff158c4e-537c-4377-a869-f9da130683f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.357442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff158c4e-537c-4377-a869-f9da130683f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.357554 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42b07b-5aec-47f5-8a46-ecfa71d11890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.357719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff158c4e-537c-4377-a869-f9da130683f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.358200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42b07b-5aec-47f5-8a46-ecfa71d11890 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.358511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42b07b-5aec-47f5-8a46-ecfa71d11890 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.358643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff158c4e-537c-4377-a869-f9da130683f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.358835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42b07b-5aec-47f5-8a46-ecfa71d11890 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.359551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42b07b-5aec-47f5-8a46-ecfa71d11890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.360261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff158c4e-537c-4377-a869-f9da130683f9 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.361469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42b07b-5aec-47f5-8a46-ecfa71d11890 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.366247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:19.371406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1032/2042] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:19.375632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff158c4e-537c-4377-a869-f9da130683f9 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.386810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff158c4e-537c-4377-a869-f9da130683f9 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.391365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1011/2043] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:19 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:19.391655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eeed81-eec3-436f-9485-a007c245e49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.392751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeed81-eec3-436f-9485-a007c245e49d admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.393111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeed81-eec3-436f-9485-a007c245e49d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.393463 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeed81-eec3-436f-9485-a007c245e49d admin admin] PATH_INFO: `/v3/projects/e060c3513c8c4c459fe9ef1697b77963/users/5b4a5836fd134ebba97249c2cae943fe/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.395484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42b07b-5aec-47f5-8a46-ecfa71d11890 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.395679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeed81-eec3-436f-9485-a007c245e49d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e060c3513c8c4c459fe9ef1697b77963, user_id=5b4a5836fd134ebba97249c2cae943fe, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.398316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeed81-eec3-436f-9485-a007c245e49d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.409776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1033/2044] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:19 2022] PUT /identity/v3/projects/e060c3513c8c4c459fe9ef1697b77963/users/5b4a5836fd134ebba97249c2cae943fe/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:19.419972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef180b-ee9c-45b7-9024-7fdbfa46f172 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.420289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef180b-ee9c-45b7-9024-7fdbfa46f172 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.420570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef180b-ee9c-45b7-9024-7fdbfa46f172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.448229 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-47ef180b-ee9c-45b7-9024-7fdbfa46f172 None None] MFA Rules not processed for user `5b4a5836fd134ebba97249c2cae943fe`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:19.449443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-47ef180b-ee9c-45b7-9024-7fdbfa46f172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:19.460002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42b07b-5aec-47f5-8a46-ecfa71d11890 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.519395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82263ca3-30ba-4640-8fae-8f9d9fa1fd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.520069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82263ca3-30ba-4640-8fae-8f9d9fa1fd79 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.520350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82263ca3-30ba-4640-8fae-8f9d9fa1fd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.520577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1034/2045] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:19.520680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82263ca3-30ba-4640-8fae-8f9d9fa1fd79 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.523516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82263ca3-30ba-4640-8fae-8f9d9fa1fd79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.525640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82263ca3-30ba-4640-8fae-8f9d9fa1fd79 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.531851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef180b-ee9c-45b7-9024-7fdbfa46f172 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.533314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-47ef180b-ee9c-45b7-9024-7fdbfa46f172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:19.550653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1012/2046] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:19 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:19.554375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82263ca3-30ba-4640-8fae-8f9d9fa1fd79 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.565279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82263ca3-30ba-4640-8fae-8f9d9fa1fd79 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.570121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5bfad-51bb-4a96-8928-0bcf62f1f42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.570812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5bfad-51bb-4a96-8928-0bcf62f1f42f admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.571104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5bfad-51bb-4a96-8928-0bcf62f1f42f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.571382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5bfad-51bb-4a96-8928-0bcf62f1f42f admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.572262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5bfad-51bb-4a96-8928-0bcf62f1f42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.574168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5bfad-51bb-4a96-8928-0bcf62f1f42f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.588914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1013/2047] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:19.591873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5bfad-51bb-4a96-8928-0bcf62f1f42f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.599318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef180b-ee9c-45b7-9024-7fdbfa46f172 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.603279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5bfad-51bb-4a96-8928-0bcf62f1f42f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.606925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457bf39-3e0b-4b61-8364-de540f10ec74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.607595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457bf39-3e0b-4b61-8364-de540f10ec74 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.607882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457bf39-3e0b-4b61-8364-de540f10ec74 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.609297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457bf39-3e0b-4b61-8364-de540f10ec74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.611205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0457bf39-3e0b-4b61-8364-de540f10ec74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:19.613100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dbc31-cf96-4fe3-b043-a1965f4948df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.613784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dbc31-cf96-4fe3-b043-a1965f4948df admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.614068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dbc31-cf96-4fe3-b043-a1965f4948df admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.614351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dbc31-cf96-4fe3-b043-a1965f4948df admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.615114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dbc31-cf96-4fe3-b043-a1965f4948df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.616766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dbc31-cf96-4fe3-b043-a1965f4948df admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.628377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1014/2048] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:19.632472 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dbc31-cf96-4fe3-b043-a1965f4948df admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.642327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dbc31-cf96-4fe3-b043-a1965f4948df admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.646973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d89a6b-f600-4f65-b2ab-d255bec3cb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.647848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d89a6b-f600-4f65-b2ab-d255bec3cb7e admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.648123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d89a6b-f600-4f65-b2ab-d255bec3cb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.648403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d89a6b-f600-4f65-b2ab-d255bec3cb7e admin admin] PATH_INFO: `/v3/projects/61b7630c1a674d67a91d49e48dcbf292/users/9046411ef5d548bbad8d3f2d778f6156/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.663670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d89a6b-f600-4f65-b2ab-d255bec3cb7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b7630c1a674d67a91d49e48dcbf292, user_id=9046411ef5d548bbad8d3f2d778f6156, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.666407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d89a6b-f600-4f65-b2ab-d255bec3cb7e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.668073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:19.671490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457bf39-3e0b-4b61-8364-de540f10ec74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.673963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457bf39-3e0b-4b61-8364-de540f10ec74 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.680240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1015/2049] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:19 2022] PUT /identity/v3/projects/61b7630c1a674d67a91d49e48dcbf292/users/9046411ef5d548bbad8d3f2d778f6156/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:19.687949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1035/2050] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:19.691816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f68c-1dd1-4f63-aed6-ef29e7ca8fe2 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.692106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f68c-1dd1-4f63-aed6-ef29e7ca8fe2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.692383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f68c-1dd1-4f63-aed6-ef29e7ca8fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.720306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b8e7f68c-1dd1-4f63-aed6-ef29e7ca8fe2 None None] MFA Rules not processed for user `9046411ef5d548bbad8d3f2d778f6156`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:19.721562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b8e7f68c-1dd1-4f63-aed6-ef29e7ca8fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:19.750848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:19.800110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1016/2051] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:19.804746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457bf39-3e0b-4b61-8364-de540f10ec74 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.816984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457bf39-3e0b-4b61-8364-de540f10ec74 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:19.887425 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952c5f5-1707-4e7f-8cf6-042d659c2c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:19.888094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952c5f5-1707-4e7f-8cf6-042d659c2c21 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:19.888385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952c5f5-1707-4e7f-8cf6-042d659c2c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:19.888662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952c5f5-1707-4e7f-8cf6-042d659c2c21 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:19.889512 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952c5f5-1707-4e7f-8cf6-042d659c2c21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:19.891947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952c5f5-1707-4e7f-8cf6-042d659c2c21 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:19.919588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1036/2052] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:19 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:19.925176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7f68c-1dd1-4f63-aed6-ef29e7ca8fe2 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:19.936637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7f68c-1dd1-4f63-aed6-ef29e7ca8fe2 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:20.002750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4eb0c1-c4e3-4e91-9cdf-15e276a1e343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:20.003599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4eb0c1-c4e3-4e91-9cdf-15e276a1e343 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:20.003984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4eb0c1-c4e3-4e91-9cdf-15e276a1e343 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:20.004322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4eb0c1-c4e3-4e91-9cdf-15e276a1e343 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:20.005092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4eb0c1-c4e3-4e91-9cdf-15e276a1e343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:20.007286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4eb0c1-c4e3-4e91-9cdf-15e276a1e343 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:20.031597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1017/2053] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:19 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:20.033284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952c5f5-1707-4e7f-8cf6-042d659c2c21 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:20.045410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952c5f5-1707-4e7f-8cf6-042d659c2c21 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:20.050963 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2540a-3efc-4be2-9282-481346d107a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:20.051627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2540a-3efc-4be2-9282-481346d107a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:20.051912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2540a-3efc-4be2-9282-481346d107a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:20.052191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2540a-3efc-4be2-9282-481346d107a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:20.052914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2540a-3efc-4be2-9282-481346d107a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:20.054520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2540a-3efc-4be2-9282-481346d107a8 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:20.065305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1037/2054] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:20.069366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4eb0c1-c4e3-4e91-9cdf-15e276a1e343 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:20.079566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4eb0c1-c4e3-4e91-9cdf-15e276a1e343 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:20.084257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30464a-37d4-4e50-bcfe-d4836aa380d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:20.085139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30464a-37d4-4e50-bcfe-d4836aa380d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:20.085424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30464a-37d4-4e50-bcfe-d4836aa380d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:20.085701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30464a-37d4-4e50-bcfe-d4836aa380d5 admin admin] PATH_INFO: `/v3/projects/f1ff98e30957440b91d6620162a00b41/users/16f995084d03463397c3b69714ed9f40/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:20.101859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30464a-37d4-4e50-bcfe-d4836aa380d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ff98e30957440b91d6620162a00b41, user_id=16f995084d03463397c3b69714ed9f40, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:20.104541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30464a-37d4-4e50-bcfe-d4836aa380d5 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:20.118839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1018/2055] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:20 2022] PUT /identity/v3/projects/f1ff98e30957440b91d6620162a00b41/users/16f995084d03463397c3b69714ed9f40/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:20.122588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2540a-3efc-4be2-9282-481346d107a8 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:20.132506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2540a-3efc-4be2-9282-481346d107a8 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:20.192054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c65058-d151-405e-8535-f274fccbcebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:20.192640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c65058-d151-405e-8535-f274fccbcebd admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:20.192926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c65058-d151-405e-8535-f274fccbcebd admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:20.193208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c65058-d151-405e-8535-f274fccbcebd admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:20.193922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c65058-d151-405e-8535-f274fccbcebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:20.195562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c65058-d151-405e-8535-f274fccbcebd admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:20.205736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1038/2056] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:20.209924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30464a-37d4-4e50-bcfe-d4836aa380d5 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:20.220796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30464a-37d4-4e50-bcfe-d4836aa380d5 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:20.225458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064318e-223b-4de3-8336-33e426e70989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:20.226346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064318e-223b-4de3-8336-33e426e70989 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:20.226633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064318e-223b-4de3-8336-33e426e70989 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:20.226951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064318e-223b-4de3-8336-33e426e70989 admin admin] PATH_INFO: `/v3/projects/f1ff98e30957440b91d6620162a00b41/users/16f995084d03463397c3b69714ed9f40/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:20.228990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064318e-223b-4de3-8336-33e426e70989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ff98e30957440b91d6620162a00b41, user_id=16f995084d03463397c3b69714ed9f40, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:20.231659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064318e-223b-4de3-8336-33e426e70989 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:20.242024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1019/2057] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:20 2022] PUT /identity/v3/projects/f1ff98e30957440b91d6620162a00b41/users/16f995084d03463397c3b69714ed9f40/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:20.252748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ad660-8c63-4aaf-9923-0d083d0aeb9a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:20.253232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ad660-8c63-4aaf-9923-0d083d0aeb9a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:20.253323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ad660-8c63-4aaf-9923-0d083d0aeb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:20.280262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2e6ad660-8c63-4aaf-9923-0d083d0aeb9a None None] MFA Rules not processed for user `16f995084d03463397c3b69714ed9f40`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:20.282093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2e6ad660-8c63-4aaf-9923-0d083d0aeb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:20.332450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:20.351260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1039/2058] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:20.361823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064318e-223b-4de3-8336-33e426e70989 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:20.363582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1064318e-223b-4de3-8336-33e426e70989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:20.421027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064318e-223b-4de3-8336-33e426e70989 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:20.424668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10f92f-9de7-4e41-852c-495934bc6f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:20.425304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10f92f-9de7-4e41-852c-495934bc6f36 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:20.425594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10f92f-9de7-4e41-852c-495934bc6f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:20.425872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10f92f-9de7-4e41-852c-495934bc6f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:20.434770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10f92f-9de7-4e41-852c-495934bc6f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:20.436200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10f92f-9de7-4e41-852c-495934bc6f36 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:20.445775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1020/2059] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:20.965636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ad660-8c63-4aaf-9923-0d083d0aeb9a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:21.032217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ad660-8c63-4aaf-9923-0d083d0aeb9a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:21.037234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:21.038106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:21.038383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:21.038654 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] PATH_INFO: `/v3/users/9046411ef5d548bbad8d3f2d778f6156` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:21.039869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9046411ef5d548bbad8d3f2d778f6156)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:21.041828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:21.087809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9046411ef5d548bbad8d3f2d778f6156'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.095332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9046411ef5d548bbad8d3f2d778f6156'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.102200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9046411ef5d548bbad8d3f2d778f6156'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.102852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9046411ef5d548bbad8d3f2d778f6156'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.114433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9046411ef5d548bbad8d3f2d778f6156'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.120335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1040/2060] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:20 2022] DELETE /identity/v3/users/9046411ef5d548bbad8d3f2d778f6156 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:21.124644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7e4a8-5e04-426c-bcff-d7c317c8eaec None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:21.125089 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7e4a8-5e04-426c-bcff-d7c317c8eaec None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:21.125196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7e4a8-5e04-426c-bcff-d7c317c8eaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:21.142938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c1a7e4a8-5e04-426c-bcff-d7c317c8eaec None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:21.144135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c1a7e4a8-5e04-426c-bcff-d7c317c8eaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:21.161365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:21.206283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1021/2061] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:21.216108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:21.217500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:21.274379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1d11b-712b-4f84-9ab2-d2e7df6ed778 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:21.278772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac18a8-0577-46e0-977a-c86c34a75ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:21.279453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac18a8-0577-46e0-977a-c86c34a75ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:21.279738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac18a8-0577-46e0-977a-c86c34a75ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:21.280016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac18a8-0577-46e0-977a-c86c34a75ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:21.290096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac18a8-0577-46e0-977a-c86c34a75ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:21.291575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac18a8-0577-46e0-977a-c86c34a75ae3 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:21.301116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1041/2062] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:21.567462 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7e4a8-5e04-426c-bcff-d7c317c8eaec None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:21.568913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c1a7e4a8-5e04-426c-bcff-d7c317c8eaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:21.582700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7e4a8-5e04-426c-bcff-d7c317c8eaec None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:21.587448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:21.588318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:21.588604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:21.588884 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] PATH_INFO: `/v3/projects/61b7630c1a674d67a91d49e48dcbf292` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:21.590592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b7630c1a674d67a91d49e48dcbf292)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:21.592816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:21.639282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b7630c1a674d67a91d49e48dcbf292', 'request_id': 'req-3ad9847a-f516-47e0-aee3-1d5325e4333e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.645880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b7630c1a674d67a91d49e48dcbf292', 'request_id': 'req-3ad9847a-f516-47e0-aee3-1d5325e4333e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.653340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b7630c1a674d67a91d49e48dcbf292', 'request_id': 'req-3ad9847a-f516-47e0-aee3-1d5325e4333e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.664210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] The token cache is being invalidate because project 61b7630c1a674d67a91d49e48dcbf292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:21.664497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.665122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.666686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1022/2063] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:21 2022] DELETE /identity/v3/projects/61b7630c1a674d67a91d49e48dcbf292 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:21.670667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac18a8-0577-46e0-977a-c86c34a75ae3 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:21.680293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac18a8-0577-46e0-977a-c86c34a75ae3 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:21.689878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:21.742495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:21.743334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:21.743608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:21.743883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] PATH_INFO: `/v3/users/16f995084d03463397c3b69714ed9f40` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:21.745055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f995084d03463397c3b69714ed9f40)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:21.747018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:21.792600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f995084d03463397c3b69714ed9f40'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.803360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f995084d03463397c3b69714ed9f40'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.810459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f995084d03463397c3b69714ed9f40'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.811321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f995084d03463397c3b69714ed9f40'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.824173 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7a106203-9b22-40bf-a3de-85aaa56fc1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f995084d03463397c3b69714ed9f40'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:21.830066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1042/2064] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:21 2022] DELETE /identity/v3/users/16f995084d03463397c3b69714ed9f40 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:22.104055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:22.106092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:22.125108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9847a-f516-47e0-aee3-1d5325e4333e admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:22.201463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:22.202380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.202713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.203033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] PATH_INFO: `/v3/projects/f1ff98e30957440b91d6620162a00b41` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.204241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ff98e30957440b91d6620162a00b41)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:22.206141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:22.253262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:22.254560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ff98e30957440b91d6620162a00b41', 'request_id': 'req-4c099746-3a18-4d29-89ee-21ba38efe54f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:22.262015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ff98e30957440b91d6620162a00b41', 'request_id': 'req-4c099746-3a18-4d29-89ee-21ba38efe54f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:22.269301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ff98e30957440b91d6620162a00b41', 'request_id': 'req-4c099746-3a18-4d29-89ee-21ba38efe54f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:22.281820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] The token cache is being invalidate because project f1ff98e30957440b91d6620162a00b41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:22.282108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:22.282760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4c099746-3a18-4d29-89ee-21ba38efe54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:22.284289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1023/2065] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:22 2022] DELETE /identity/v3/projects/f1ff98e30957440b91d6620162a00b41 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:22.290630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46721e9-57c4-4530-a34c-cce5cc7564fa None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.291139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46721e9-57c4-4530-a34c-cce5cc7564fa None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.291251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46721e9-57c4-4530-a34c-cce5cc7564fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.308882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-f46721e9-57c4-4530-a34c-cce5cc7564fa None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:22.310118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f46721e9-57c4-4530-a34c-cce5cc7564fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:22.368044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1043/2066] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:22.375939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49bd86-4446-4659-8087-55d3bfc7370f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.376233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49bd86-4446-4659-8087-55d3bfc7370f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.376547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49bd86-4446-4659-8087-55d3bfc7370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.394288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9a49bd86-4446-4659-8087-55d3bfc7370f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:22.395515 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9a49bd86-4446-4659-8087-55d3bfc7370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:22.403321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1024/2067] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:22.407722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46721e9-57c4-4530-a34c-cce5cc7564fa None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:22.417623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46721e9-57c4-4530-a34c-cce5cc7564fa None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:22.422920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b21f60-93bb-4582-89b9-488f59ec0fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:22.423567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b21f60-93bb-4582-89b9-488f59ec0fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.423852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b21f60-93bb-4582-89b9-488f59ec0fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.424134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b21f60-93bb-4582-89b9-488f59ec0fc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.424856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b21f60-93bb-4582-89b9-488f59ec0fc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:22.425166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b21f60-93bb-4582-89b9-488f59ec0fc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:22.426785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b21f60-93bb-4582-89b9-488f59ec0fc8 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:22.438199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1044/2068] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:22.442341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49bd86-4446-4659-8087-55d3bfc7370f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:22.451753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49bd86-4446-4659-8087-55d3bfc7370f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:22.456891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e37b35-6c5d-437e-8f02-024934d8ccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:22.457544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e37b35-6c5d-437e-8f02-024934d8ccd5 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.457833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e37b35-6c5d-437e-8f02-024934d8ccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.458110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e37b35-6c5d-437e-8f02-024934d8ccd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.458873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e37b35-6c5d-437e-8f02-024934d8ccd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:22.460768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e37b35-6c5d-437e-8f02-024934d8ccd5 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:22.486436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1025/2069] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:22 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:22.490552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b21f60-93bb-4582-89b9-488f59ec0fc8 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:22.500584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b21f60-93bb-4582-89b9-488f59ec0fc8 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:22.517780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:22.558368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1486cf-7dd5-4889-a32d-9f8e4d9ef105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:22.558984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1486cf-7dd5-4889-a32d-9f8e4d9ef105 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.559273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1486cf-7dd5-4889-a32d-9f8e4d9ef105 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.559553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1486cf-7dd5-4889-a32d-9f8e4d9ef105 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.560239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1486cf-7dd5-4889-a32d-9f8e4d9ef105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:22.562185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1486cf-7dd5-4889-a32d-9f8e4d9ef105 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:22.581612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1045/2070] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:22 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:22.585656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e37b35-6c5d-437e-8f02-024934d8ccd5 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:22.596750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e37b35-6c5d-437e-8f02-024934d8ccd5 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:22.601411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad916a-5bb5-418c-83d9-0660f062c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:22.602063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad916a-5bb5-418c-83d9-0660f062c71e admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.602351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad916a-5bb5-418c-83d9-0660f062c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.602631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad916a-5bb5-418c-83d9-0660f062c71e admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.603408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad916a-5bb5-418c-83d9-0660f062c71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:22.604995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad916a-5bb5-418c-83d9-0660f062c71e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:22.616323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1026/2071] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:22.620162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1486cf-7dd5-4889-a32d-9f8e4d9ef105 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:22.630053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1486cf-7dd5-4889-a32d-9f8e4d9ef105 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:22.634759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28bb80-962f-45d8-a46f-3487fa212374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:22.635408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28bb80-962f-45d8-a46f-3487fa212374 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.635693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28bb80-962f-45d8-a46f-3487fa212374 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.635972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28bb80-962f-45d8-a46f-3487fa212374 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.636724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28bb80-962f-45d8-a46f-3487fa212374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:22.638323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28bb80-962f-45d8-a46f-3487fa212374 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:22.648619 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1046/2072] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:22.652445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad916a-5bb5-418c-83d9-0660f062c71e admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:22.670784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad916a-5bb5-418c-83d9-0660f062c71e admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:22.675444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cf0e2-2c8e-4ca6-aa00-c0e8f78ac4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:22.676330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cf0e2-2c8e-4ca6-aa00-c0e8f78ac4be admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.676618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cf0e2-2c8e-4ca6-aa00-c0e8f78ac4be admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.676894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cf0e2-2c8e-4ca6-aa00-c0e8f78ac4be admin admin] PATH_INFO: `/v3/projects/aea85c175f6e445eb1d2a16aa8e0fec7/users/445dfdeb072e43649d05f738042c6df6/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.691715 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1cf0e2-2c8e-4ca6-aa00-c0e8f78ac4be admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea85c175f6e445eb1d2a16aa8e0fec7, user_id=445dfdeb072e43649d05f738042c6df6, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:22.694522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1cf0e2-2c8e-4ca6-aa00-c0e8f78ac4be admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:22.707502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1027/2073] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:22 2022] PUT /identity/v3/projects/aea85c175f6e445eb1d2a16aa8e0fec7/users/445dfdeb072e43649d05f738042c6df6/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:22.719585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd0741-593f-4fe2-aa73-1500b2e35868 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.719875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd0741-593f-4fe2-aa73-1500b2e35868 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.720157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd0741-593f-4fe2-aa73-1500b2e35868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.745730 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-18fd0741-593f-4fe2-aa73-1500b2e35868 None None] MFA Rules not processed for user `445dfdeb072e43649d05f738042c6df6`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:22.746982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-18fd0741-593f-4fe2-aa73-1500b2e35868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:22.813009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1047/2074] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:22.825383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cf0e2-2c8e-4ca6-aa00-c0e8f78ac4be admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:22.825488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-dd1cf0e2-2c8e-4ca6-aa00-c0e8f78ac4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:22.880858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cf0e2-2c8e-4ca6-aa00-c0e8f78ac4be admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:22.884433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11c6c8-2797-45ec-a67c-c9c95aa68cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:22.885063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11c6c8-2797-45ec-a67c-c9c95aa68cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.885344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11c6c8-2797-45ec-a67c-c9c95aa68cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.885621 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11c6c8-2797-45ec-a67c-c9c95aa68cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.894480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11c6c8-2797-45ec-a67c-c9c95aa68cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:22.895926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11c6c8-2797-45ec-a67c-c9c95aa68cc2 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:22.904488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1028/2075] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:22.923742 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd0741-593f-4fe2-aa73-1500b2e35868 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:22.985701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd0741-593f-4fe2-aa73-1500b2e35868 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:22.990344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:22.991235 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:22.991513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:22.991786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] PATH_INFO: `/v3/users/445dfdeb072e43649d05f738042c6df6` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:22.992997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=445dfdeb072e43649d05f738042c6df6)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:22.994949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:23.026636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:23.041737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '445dfdeb072e43649d05f738042c6df6'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.049778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '445dfdeb072e43649d05f738042c6df6'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.057408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '445dfdeb072e43649d05f738042c6df6'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.058067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '445dfdeb072e43649d05f738042c6df6'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.070767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '445dfdeb072e43649d05f738042c6df6'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.076900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1048/2076] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:22 2022] DELETE /identity/v3/users/445dfdeb072e43649d05f738042c6df6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:23.081107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749161cd-a9c3-41b8-b193-fd6eba5e906b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:23.081475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749161cd-a9c3-41b8-b193-fd6eba5e906b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:23.081844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749161cd-a9c3-41b8-b193-fd6eba5e906b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:23.100443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-749161cd-a9c3-41b8-b193-fd6eba5e906b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:23.101649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-749161cd-a9c3-41b8-b193-fd6eba5e906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:23.174143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1029/2077] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:23.187783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:23.190078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:23.307641 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeece18a-7193-46a8-bf4b-b32b0364c224 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:23.312105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ec093-ce5c-4c7c-9c51-2921049aff91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:23.312832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ec093-ce5c-4c7c-9c51-2921049aff91 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:23.313188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ec093-ce5c-4c7c-9c51-2921049aff91 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:23.313530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ec093-ce5c-4c7c-9c51-2921049aff91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:23.325596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ec093-ce5c-4c7c-9c51-2921049aff91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:23.327846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ec093-ce5c-4c7c-9c51-2921049aff91 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:23.342599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1049/2078] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:23.380501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749161cd-a9c3-41b8-b193-fd6eba5e906b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:23.381978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-749161cd-a9c3-41b8-b193-fd6eba5e906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:23.416054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:23.479696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749161cd-a9c3-41b8-b193-fd6eba5e906b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:23.483910 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2beb92-7ae3-4a5f-bd03-93b3af5f53b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:23.484820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2beb92-7ae3-4a5f-bd03-93b3af5f53b1 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:23.485147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2beb92-7ae3-4a5f-bd03-93b3af5f53b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:23.485778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2beb92-7ae3-4a5f-bd03-93b3af5f53b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:23.487949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8d2beb92-7ae3-4a5f-bd03-93b3af5f53b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:23.585480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2beb92-7ae3-4a5f-bd03-93b3af5f53b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:23.587418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2beb92-7ae3-4a5f-bd03-93b3af5f53b1 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:23.601446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1030/2079] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:23.700189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ec093-ce5c-4c7c-9c51-2921049aff91 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:23.701126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c11ec093-ce5c-4c7c-9c51-2921049aff91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:23.724897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ec093-ce5c-4c7c-9c51-2921049aff91 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:23.742413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:23.743359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:23.743726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:23.743949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] PATH_INFO: `/v3/projects/aea85c175f6e445eb1d2a16aa8e0fec7` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:23.745588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea85c175f6e445eb1d2a16aa8e0fec7)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:23.749459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:23.854759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea85c175f6e445eb1d2a16aa8e0fec7', 'request_id': 'req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.867057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea85c175f6e445eb1d2a16aa8e0fec7', 'request_id': 'req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.894315 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea85c175f6e445eb1d2a16aa8e0fec7', 'request_id': 'req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.918348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] The token cache is being invalidate because project aea85c175f6e445eb1d2a16aa8e0fec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:23.918655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.923262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-02458f30-ae1c-4af1-ab57-90bd14d7f8ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:23.927179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1050/2080] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:23 2022] DELETE /identity/v3/projects/aea85c175f6e445eb1d2a16aa8e0fec7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:23.932203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1114e3-19c0-41b2-9432-f83c0be4d27c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:23.932732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1114e3-19c0-41b2-9432-f83c0be4d27c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:23.932930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1114e3-19c0-41b2-9432-f83c0be4d27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:23.954527 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8c1114e3-19c0-41b2-9432-f83c0be4d27c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:23.955808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8c1114e3-19c0-41b2-9432-f83c0be4d27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:24.024644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1031/2081] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:24.032261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738f23f-172c-4fcd-aca5-dc41bcdc0e3e None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.032543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738f23f-172c-4fcd-aca5-dc41bcdc0e3e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.032819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738f23f-172c-4fcd-aca5-dc41bcdc0e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.051670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b738f23f-172c-4fcd-aca5-dc41bcdc0e3e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:24.052866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b738f23f-172c-4fcd-aca5-dc41bcdc0e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:24.063233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1051/2082] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:24.067938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1114e3-19c0-41b2-9432-f83c0be4d27c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:24.078264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1114e3-19c0-41b2-9432-f83c0be4d27c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:24.083117 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd74f3-d9d1-4705-9ffb-e5196376e0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:24.083781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd74f3-d9d1-4705-9ffb-e5196376e0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.084064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd74f3-d9d1-4705-9ffb-e5196376e0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.084343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd74f3-d9d1-4705-9ffb-e5196376e0f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.085093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd74f3-d9d1-4705-9ffb-e5196376e0f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:24.085399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd74f3-d9d1-4705-9ffb-e5196376e0f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:24.087010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd74f3-d9d1-4705-9ffb-e5196376e0f9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:24.099455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1032/2083] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:24.104218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738f23f-172c-4fcd-aca5-dc41bcdc0e3e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:24.115202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738f23f-172c-4fcd-aca5-dc41bcdc0e3e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:24.120103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7494a17-23db-4a6a-ab3f-61b49e4ded0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:24.120764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7494a17-23db-4a6a-ab3f-61b49e4ded0b admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.121052 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7494a17-23db-4a6a-ab3f-61b49e4ded0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.121325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7494a17-23db-4a6a-ab3f-61b49e4ded0b admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.122036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7494a17-23db-4a6a-ab3f-61b49e4ded0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:24.124528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7494a17-23db-4a6a-ab3f-61b49e4ded0b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:24.149462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1052/2084] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:24 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:24.153892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd74f3-d9d1-4705-9ffb-e5196376e0f9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:24.164245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd74f3-d9d1-4705-9ffb-e5196376e0f9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:24.216675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:24.223309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d5677-fbc6-4a1e-9c0f-2d25b12a5f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:24.224050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d5677-fbc6-4a1e-9c0f-2d25b12a5f9a admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.224377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d5677-fbc6-4a1e-9c0f-2d25b12a5f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.224673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d5677-fbc6-4a1e-9c0f-2d25b12a5f9a admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.225543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d5677-fbc6-4a1e-9c0f-2d25b12a5f9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:24.227707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d5677-fbc6-4a1e-9c0f-2d25b12a5f9a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:24.250279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1033/2085] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:24 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:24.254438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7494a17-23db-4a6a-ab3f-61b49e4ded0b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:24.261197 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:24.265345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7494a17-23db-4a6a-ab3f-61b49e4ded0b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:24.273142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1d2e3-0182-4c81-b2c5-d181420e3164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:24.273974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1d2e3-0182-4c81-b2c5-d181420e3164 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.274433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1d2e3-0182-4c81-b2c5-d181420e3164 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.274740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1d2e3-0182-4c81-b2c5-d181420e3164 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.276090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1d2e3-0182-4c81-b2c5-d181420e3164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:24.278513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1d2e3-0182-4c81-b2c5-d181420e3164 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:24.290911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1053/2086] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:24.295203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d5677-fbc6-4a1e-9c0f-2d25b12a5f9a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:24.306452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d5677-fbc6-4a1e-9c0f-2d25b12a5f9a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:24.312644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7fb7e-f6cc-4e08-8176-c8b2e0c9ce81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:24.313408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7fb7e-f6cc-4e08-8176-c8b2e0c9ce81 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.313683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7fb7e-f6cc-4e08-8176-c8b2e0c9ce81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.313959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7fb7e-f6cc-4e08-8176-c8b2e0c9ce81 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.314677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7fb7e-f6cc-4e08-8176-c8b2e0c9ce81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:24.316298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7fb7e-f6cc-4e08-8176-c8b2e0c9ce81 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:24.327724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1034/2087] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:24.331670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1d2e3-0182-4c81-b2c5-d181420e3164 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:24.341539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1d2e3-0182-4c81-b2c5-d181420e3164 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:24.346643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b27c2f-91db-4b2c-9676-33e939eba1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:24.347543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b27c2f-91db-4b2c-9676-33e939eba1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.347820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b27c2f-91db-4b2c-9676-33e939eba1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.348119 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b27c2f-91db-4b2c-9676-33e939eba1d5 admin admin] PATH_INFO: `/v3/projects/047da195e943441fabfc3ab9d7eb420f/users/27737b6037514a908fe209f41b22e293/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.364541 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b27c2f-91db-4b2c-9676-33e939eba1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=047da195e943441fabfc3ab9d7eb420f, user_id=27737b6037514a908fe209f41b22e293, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:24.367230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b27c2f-91db-4b2c-9676-33e939eba1d5 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:24.380279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1054/2088] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:24 2022] PUT /identity/v3/projects/047da195e943441fabfc3ab9d7eb420f/users/27737b6037514a908fe209f41b22e293/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:24.390549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf8943-e109-47f5-b396-0bdb6815f427 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.390866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf8943-e109-47f5-b396-0bdb6815f427 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.391142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf8943-e109-47f5-b396-0bdb6815f427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.419623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0aaf8943-e109-47f5-b396-0bdb6815f427 None None] MFA Rules not processed for user `27737b6037514a908fe209f41b22e293`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:24.421446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0aaf8943-e109-47f5-b396-0bdb6815f427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:24.497167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1035/2089] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:24.508976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b27c2f-91db-4b2c-9676-33e939eba1d5 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:24.510701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-82b27c2f-91db-4b2c-9676-33e939eba1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:24.583596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b27c2f-91db-4b2c-9676-33e939eba1d5 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:24.587530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9a9a2-3cfa-4800-ba74-c93e5da4f87c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:24.588174 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9a9a2-3cfa-4800-ba74-c93e5da4f87c service glance] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.588456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9a9a2-3cfa-4800-ba74-c93e5da4f87c service glance] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.588733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9a9a2-3cfa-4800-ba74-c93e5da4f87c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.598803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9a9a2-3cfa-4800-ba74-c93e5da4f87c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:24.600402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9a9a2-3cfa-4800-ba74-c93e5da4f87c service glance] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:24.600957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf8943-e109-47f5-b396-0bdb6815f427 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:24.602403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0aaf8943-e109-47f5-b396-0bdb6815f427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:24.618839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1055/2090] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:24.667463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf8943-e109-47f5-b396-0bdb6815f427 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:24.671657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:24.672223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:24.672503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:24.672775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:24.677828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:24.741436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:24.742875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:24.752549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1036/2091] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:28.896896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297d12a-fad5-4a20-b428-0f0b0a3ce14c None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:28.897435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297d12a-fad5-4a20-b428-0f0b0a3ce14c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:28.897708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297d12a-fad5-4a20-b428-0f0b0a3ce14c None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:28.898691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:28.899573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1056/2092] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:51:28.900219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:28.912908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6004a-190f-482f-a749-bc73ee2f8e69 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:28.913192 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6004a-190f-482f-a749-bc73ee2f8e69 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:28.913476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6004a-190f-482f-a749-bc73ee2f8e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:28.924046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:28.935889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-83d6004a-190f-482f-a749-bc73ee2f8e69 None None] MFA Rules not processed for user `a786a95c48c348dbba269e0e33265271`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:28.937154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-83d6004a-190f-482f-a749-bc73ee2f8e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:28.994495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48019e6e-01ec-4e17-a155-5edf021e3be8 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.006186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.007132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.007421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.007721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] PATH_INFO: `/v3/users/fe3e0fe92cf544aa80e4e87877706f88` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.008932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3e0fe92cf544aa80e4e87877706f88)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.010906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.063980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1057/2093] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:29.065059 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6004a-190f-482f-a749-bc73ee2f8e69 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.066517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-83d6004a-190f-482f-a749-bc73ee2f8e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.080646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6004a-190f-482f-a749-bc73ee2f8e69 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.085596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.086454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.086768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.087045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] PATH_INFO: `/v3/users/c1d2305db6284067bfb37c3b1bf198fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.088003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3e0fe92cf544aa80e4e87877706f88'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.088255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d2305db6284067bfb37c3b1bf198fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.090176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.100088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3e0fe92cf544aa80e4e87877706f88'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.107476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3e0fe92cf544aa80e4e87877706f88'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.108568 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3e0fe92cf544aa80e4e87877706f88'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.111067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3e0fe92cf544aa80e4e87877706f88'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.145997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d2305db6284067bfb37c3b1bf198fa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.152062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1037/2094] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:28 2022] DELETE /identity/v3/users/fe3e0fe92cf544aa80e4e87877706f88 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:29.152944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.154399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.164681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d2305db6284067bfb37c3b1bf198fa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.172480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d2305db6284067bfb37c3b1bf198fa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.173165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d2305db6284067bfb37c3b1bf198fa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.174563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:29.192365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2c743209-1851-4410-b129-5f2dd147e95b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d2305db6284067bfb37c3b1bf198fa'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.198939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1058/2095] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:29 2022] DELETE /identity/v3/users/c1d2305db6284067bfb37c3b1bf198fa => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:29.200551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137722-6830-4865-9716-1559d521cd28 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.200858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137722-6830-4865-9716-1559d521cd28 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.201139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137722-6830-4865-9716-1559d521cd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.220928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-68137722-6830-4865-9716-1559d521cd28 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:29.222286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-68137722-6830-4865-9716-1559d521cd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.239879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fef79a-fde6-4386-9ee4-c8182ad41a5b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.283086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:29.297162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1059/2096] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:29.298583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137722-6830-4865-9716-1559d521cd28 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.303504 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-68137722-6830-4865-9716-1559d521cd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.320585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137722-6830-4865-9716-1559d521cd28 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.325785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.326661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.326802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.327085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.327409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] PATH_INFO: `/v3/users/3ac2d23cba384a5cab550ebae9bf525b` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.327637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.327985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.328199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.328636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac2d23cba384a5cab550ebae9bf525b)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.330380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.330564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.342927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.344503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.356063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1038/2097] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:29.356886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.359287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.408150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac2d23cba384a5cab550ebae9bf525b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.423432 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac2d23cba384a5cab550ebae9bf525b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.430632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac2d23cba384a5cab550ebae9bf525b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.431907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac2d23cba384a5cab550ebae9bf525b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.437273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fc74f-bfe1-4b8b-b278-bfc8df33ea9a service placement] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.441441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65df7b9-ac60-4b84-86ac-f8a9a2155eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.442094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65df7b9-ac60-4b84-86ac-f8a9a2155eee service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.442381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65df7b9-ac60-4b84-86ac-f8a9a2155eee service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.442660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65df7b9-ac60-4b84-86ac-f8a9a2155eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.452929 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65df7b9-ac60-4b84-86ac-f8a9a2155eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.454249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65df7b9-ac60-4b84-86ac-f8a9a2155eee service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.457140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ee1d98f2-63d2-49e5-a911-820b1a4786d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac2d23cba384a5cab550ebae9bf525b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.464836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1039/2098] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:29.465031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1060/2099] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:29 2022] DELETE /identity/v3/users/3ac2d23cba384a5cab550ebae9bf525b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:29.465577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65df7b9-ac60-4b84-86ac-f8a9a2155eee service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.469536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fee9c-8be9-42e7-96ab-b692c22c4ced None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.469705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fee9c-8be9-42e7-96ab-b692c22c4ced None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.469936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fee9c-8be9-42e7-96ab-b692c22c4ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.475842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b65df7b9-ac60-4b84-86ac-f8a9a2155eee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.489490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ff5fee9c-8be9-42e7-96ab-b692c22c4ced None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:29.490774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ff5fee9c-8be9-42e7-96ab-b692c22c4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.492500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65df7b9-ac60-4b84-86ac-f8a9a2155eee service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.562921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:29.568176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.569181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.569530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.569848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] PATH_INFO: `/v3/projects/a8fb15aaa4ac4d55a68b821f99bbffd7` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.571072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8fb15aaa4ac4d55a68b821f99bbffd7)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.573415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.580725 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1061/2100] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:29.589922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fee9c-8be9-42e7-96ab-b692c22c4ced None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.591367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ff5fee9c-8be9-42e7-96ab-b692c22c4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.634817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8fb15aaa4ac4d55a68b821f99bbffd7', 'request_id': 'req-ef213fcc-e7a8-4915-bd98-3384929fb376'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.641579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8fb15aaa4ac4d55a68b821f99bbffd7', 'request_id': 'req-ef213fcc-e7a8-4915-bd98-3384929fb376'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.660841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8fb15aaa4ac4d55a68b821f99bbffd7', 'request_id': 'req-ef213fcc-e7a8-4915-bd98-3384929fb376'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.664421 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fee9c-8be9-42e7-96ab-b692c22c4ced None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.670362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:29.676724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] The token cache is being invalidate because project a8fb15aaa4ac4d55a68b821f99bbffd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:29.677571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.682255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ef213fcc-e7a8-4915-bd98-3384929fb376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.684749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1040/2101] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:29 2022] DELETE /identity/v3/projects/a8fb15aaa4ac4d55a68b821f99bbffd7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:29.691371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916133ac-e611-46ce-b105-d48b41f1b6f3 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.692150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916133ac-e611-46ce-b105-d48b41f1b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.692150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916133ac-e611-46ce-b105-d48b41f1b6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.714541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-916133ac-e611-46ce-b105-d48b41f1b6f3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:29.716009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-916133ac-e611-46ce-b105-d48b41f1b6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.735226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d324725-41bf-4f4d-8bd7-f19d4e1ec22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.735969 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d324725-41bf-4f4d-8bd7-f19d4e1ec22f service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.736317 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d324725-41bf-4f4d-8bd7-f19d4e1ec22f service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.736663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d324725-41bf-4f4d-8bd7-f19d4e1ec22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.738561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2d324725-41bf-4f4d-8bd7-f19d4e1ec22f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.796768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1041/2102] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:29.798352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916133ac-e611-46ce-b105-d48b41f1b6f3 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.800509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-916133ac-e611-46ce-b105-d48b41f1b6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.813580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d324725-41bf-4f4d-8bd7-f19d4e1ec22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.814620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916133ac-e611-46ce-b105-d48b41f1b6f3 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.815024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d324725-41bf-4f4d-8bd7-f19d4e1ec22f service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.821340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.822502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.822801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.823088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] PATH_INFO: `/v3/projects/975da5681ab94b11b331cf7834bae144` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.824374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=975da5681ab94b11b331cf7834bae144)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.825294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1062/2103] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:29.826444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.826573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb9a09-cd8e-4ba4-bf46-1883c9898d7a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.826793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb9a09-cd8e-4ba4-bf46-1883c9898d7a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.827107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb9a09-cd8e-4ba4-bf46-1883c9898d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.845595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-dcfb9a09-cd8e-4ba4-bf46-1883c9898d7a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:29.846855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-dcfb9a09-cd8e-4ba4-bf46-1883c9898d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.861940 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1063/2104] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:29.867120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb9a09-cd8e-4ba4-bf46-1883c9898d7a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.877758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb9a09-cd8e-4ba4-bf46-1883c9898d7a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.896982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '975da5681ab94b11b331cf7834bae144', 'request_id': 'req-79b05cfd-fce3-4b1d-8820-6501456fd3f7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.906444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '975da5681ab94b11b331cf7834bae144', 'request_id': 'req-79b05cfd-fce3-4b1d-8820-6501456fd3f7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.916660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '975da5681ab94b11b331cf7834bae144', 'request_id': 'req-79b05cfd-fce3-4b1d-8820-6501456fd3f7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.930920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] The token cache is being invalidate because project 975da5681ab94b11b331cf7834bae144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:29.931680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.932695 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:29.934660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:29.935153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1042/2105] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:29 2022] DELETE /identity/v3/projects/975da5681ab94b11b331cf7834bae144 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:29.938956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.940532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.945387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.946061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.946344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.946625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.947443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:29.947752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.949417 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.958842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b05cfd-fce3-4b1d-8820-6501456fd3f7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.966380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1064/2106] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:29.967356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:29.969575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.969709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:29.970189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.970547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.970935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] PATH_INFO: `/v3/users/14d980b05fcd4f7cbbe7f1a3e5551ca4` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.972292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d980b05fcd4f7cbbe7f1a3e5551ca4)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.974262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:29.987935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379563cc-33ce-44f1-ae96-84913cedaf8e admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:29.994615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68400240-d7ef-4907-a3dc-1d50d4828e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:29.995323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68400240-d7ef-4907-a3dc-1d50d4828e08 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:29.995613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68400240-d7ef-4907-a3dc-1d50d4828e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:29.995894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68400240-d7ef-4907-a3dc-1d50d4828e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:29.996591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68400240-d7ef-4907-a3dc-1d50d4828e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:29.998534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68400240-d7ef-4907-a3dc-1d50d4828e08 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.033007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1065/2107] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:29 2022] POST /identity/v3/projects => generated 391 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:30.038123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68400240-d7ef-4907-a3dc-1d50d4828e08 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.049918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d980b05fcd4f7cbbe7f1a3e5551ca4'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.062128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d980b05fcd4f7cbbe7f1a3e5551ca4'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.072911 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d980b05fcd4f7cbbe7f1a3e5551ca4'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.073654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d980b05fcd4f7cbbe7f1a3e5551ca4'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.074336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d980b05fcd4f7cbbe7f1a3e5551ca4'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.091550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:30.107908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1043/2108] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:29 2022] DELETE /identity/v3/users/14d980b05fcd4f7cbbe7f1a3e5551ca4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:30.140623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68400240-d7ef-4907-a3dc-1d50d4828e08 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:30.153996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.156204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:30.215730 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56d277-07bb-450c-9e87-ebed87c58e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:30.216892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56d277-07bb-450c-9e87-ebed87c58e14 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.216892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56d277-07bb-450c-9e87-ebed87c58e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.217129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56d277-07bb-450c-9e87-ebed87c58e14 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.218402 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56d277-07bb-450c-9e87-ebed87c58e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:30.220428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56d277-07bb-450c-9e87-ebed87c58e14 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.244201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1066/2109] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:30 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:30.248920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56d277-07bb-450c-9e87-ebed87c58e14 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.250628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ce56d277-07bb-450c-9e87-ebed87c58e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:30.264001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56d277-07bb-450c-9e87-ebed87c58e14 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:30.268641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a8b6c-5a17-4e43-af17-77ffa91b2b34 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:30.269819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7857ab-477d-4f68-bc47-589acb838e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:30.270502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7857ab-477d-4f68-bc47-589acb838e9f admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.271342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7857ab-477d-4f68-bc47-589acb838e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.271625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7857ab-477d-4f68-bc47-589acb838e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.272732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7857ab-477d-4f68-bc47-589acb838e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:30.274324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:30.274500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7857ab-477d-4f68-bc47-589acb838e9f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.275281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.275423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.276077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] PATH_INFO: `/v3/projects/e8716a5e56ad4fbf864aa44b7932cf50` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.277098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8716a5e56ad4fbf864aa44b7932cf50)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:30.279669 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.294804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1067/2110] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:30.295935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7857ab-477d-4f68-bc47-589acb838e9f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.307006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7857ab-477d-4f68-bc47-589acb838e9f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:30.315487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de29bd1-d20a-4f1d-96e7-efcb498ce438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:30.316257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de29bd1-d20a-4f1d-96e7-efcb498ce438 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.316582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de29bd1-d20a-4f1d-96e7-efcb498ce438 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.316920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de29bd1-d20a-4f1d-96e7-efcb498ce438 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.317944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de29bd1-d20a-4f1d-96e7-efcb498ce438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:30.319696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de29bd1-d20a-4f1d-96e7-efcb498ce438 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.334485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1068/2111] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:30.335494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de29bd1-d20a-4f1d-96e7-efcb498ce438 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.342389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8716a5e56ad4fbf864aa44b7932cf50', 'request_id': 'req-deb098bd-6373-4870-86d2-d1f570184366'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.346258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de29bd1-d20a-4f1d-96e7-efcb498ce438 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:30.350903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8716a5e56ad4fbf864aa44b7932cf50', 'request_id': 'req-deb098bd-6373-4870-86d2-d1f570184366'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.364476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8716a5e56ad4fbf864aa44b7932cf50', 'request_id': 'req-deb098bd-6373-4870-86d2-d1f570184366'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.377945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] The token cache is being invalidate because project e8716a5e56ad4fbf864aa44b7932cf50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:30.378334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.379122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.380990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1044/2112] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:30 2022] DELETE /identity/v3/projects/e8716a5e56ad4fbf864aa44b7932cf50 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:30.384556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.386036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:30.394066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:30.420299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928eab4-208e-4888-bf97-77b2ad7fcb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:30.421241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928eab4-208e-4888-bf97-77b2ad7fcb31 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.421605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928eab4-208e-4888-bf97-77b2ad7fcb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.421892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928eab4-208e-4888-bf97-77b2ad7fcb31 admin admin] PATH_INFO: `/v3/projects/f2ed62f01ef846569c069e52fcfee36a/users/f8fcb09990db433e8ac0af3de37bdbc7/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.439572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928eab4-208e-4888-bf97-77b2ad7fcb31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ed62f01ef846569c069e52fcfee36a, user_id=f8fcb09990db433e8ac0af3de37bdbc7, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:30.443987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928eab4-208e-4888-bf97-77b2ad7fcb31 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.469545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1069/2113] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:30 2022] PUT /identity/v3/projects/f2ed62f01ef846569c069e52fcfee36a/users/f8fcb09990db433e8ac0af3de37bdbc7/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:30.473511 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928eab4-208e-4888-bf97-77b2ad7fcb31 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.473511 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e928eab4-208e-4888-bf97-77b2ad7fcb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:30.481647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb098bd-6373-4870-86d2-d1f570184366 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:30.518434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:30.555086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:30.556010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.556246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.556518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] PATH_INFO: `/v3/users/0b40d08d72eb4975b763d4322fc883b7` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.558175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b40d08d72eb4975b763d4322fc883b7)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:30.560263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.560978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928eab4-208e-4888-bf97-77b2ad7fcb31 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:30.567031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:30.567298 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.567573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.567855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] PATH_INFO: `/v3/projects/51c3d83d26bc486f857ae4ffa7894bd3` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.569051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c3d83d26bc486f857ae4ffa7894bd3)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:30.570997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.614413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b40d08d72eb4975b763d4322fc883b7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.620953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c3d83d26bc486f857ae4ffa7894bd3', 'request_id': 'req-4cd2caa5-b936-4da8-bcea-b66393c45e81'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.622253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b40d08d72eb4975b763d4322fc883b7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.628553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c3d83d26bc486f857ae4ffa7894bd3', 'request_id': 'req-4cd2caa5-b936-4da8-bcea-b66393c45e81'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.630286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b40d08d72eb4975b763d4322fc883b7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.631095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b40d08d72eb4975b763d4322fc883b7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.631751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-82f5264b-2860-4048-a8dd-856722b46643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b40d08d72eb4975b763d4322fc883b7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.636601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c3d83d26bc486f857ae4ffa7894bd3', 'request_id': 'req-4cd2caa5-b936-4da8-bcea-b66393c45e81'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.652431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1045/2114] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:30 2022] DELETE /identity/v3/users/0b40d08d72eb4975b763d4322fc883b7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:30.653895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] The token cache is being invalidate because project 51c3d83d26bc486f857ae4ffa7894bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:30.654171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.655042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.656627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1070/2115] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:30 2022] DELETE /identity/v3/projects/51c3d83d26bc486f857ae4ffa7894bd3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:30.659109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ba0fa-85e5-496f-a879-f875b7ac492e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.659414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ba0fa-85e5-496f-a879-f875b7ac492e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.659994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ba0fa-85e5-496f-a879-f875b7ac492e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.660814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.662679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:30.681364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-3d4ba0fa-85e5-496f-a879-f875b7ac492e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:30.682379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3d4ba0fa-85e5-496f-a879-f875b7ac492e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:30.725878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:30.750758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2caa5-b936-4da8-bcea-b66393c45e81 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:30.756098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1046/2116] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:30.757896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:30.758914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.759204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.759485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] PATH_INFO: `/v3/users/5b4a5836fd134ebba97249c2cae943fe` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.760862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4a5836fd134ebba97249c2cae943fe)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:30.763177 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.768682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ba0fa-85e5-496f-a879-f875b7ac492e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.771289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3d4ba0fa-85e5-496f-a879-f875b7ac492e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:30.827842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4a5836fd134ebba97249c2cae943fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.838592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4a5836fd134ebba97249c2cae943fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.848607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4a5836fd134ebba97249c2cae943fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.849338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4a5836fd134ebba97249c2cae943fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.853980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ba0fa-85e5-496f-a879-f875b7ac492e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:30.859358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a61e72-7055-46e2-b858-80e3f5b1af65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:30.860010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a61e72-7055-46e2-b858-80e3f5b1af65 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:30.860303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a61e72-7055-46e2-b858-80e3f5b1af65 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:30.860582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a61e72-7055-46e2-b858-80e3f5b1af65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:30.863207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4a5836fd134ebba97249c2cae943fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:30.868324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:30.870917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a61e72-7055-46e2-b858-80e3f5b1af65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:30.872361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a61e72-7055-46e2-b858-80e3f5b1af65 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:30.874551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1071/2117] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:30 2022] DELETE /identity/v3/users/5b4a5836fd134ebba97249c2cae943fe => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:30.884854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1047/2118] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:30.983022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:30.985655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:31.082455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e9f6c-b6ac-4fb9-9575-fbd039961145 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.090122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:31.091050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.091333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.091611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] PATH_INFO: `/v3/projects/0aa31bf0a6fb4601a08d93ade0d4d9f8` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.092857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa31bf0a6fb4601a08d93ade0d4d9f8)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:31.094872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:31.162360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa31bf0a6fb4601a08d93ade0d4d9f8', 'request_id': 'req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.170365 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa31bf0a6fb4601a08d93ade0d4d9f8', 'request_id': 'req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.180918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa31bf0a6fb4601a08d93ade0d4d9f8', 'request_id': 'req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.196515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] The token cache is being invalidate because project 0aa31bf0a6fb4601a08d93ade0d4d9f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:31.196515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.197254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.199937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1072/2119] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:30 2022] DELETE /identity/v3/projects/0aa31bf0a6fb4601a08d93ade0d4d9f8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:31.208001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a4bda-110f-4f20-9387-52176661ce02 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.208491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a4bda-110f-4f20-9387-52176661ce02 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.208596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a4bda-110f-4f20-9387-52176661ce02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.219823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:31.221617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:31.227826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:31.246807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-626a4bda-110f-4f20-9387-52176661ce02 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:31.247987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-626a4bda-110f-4f20-9387-52176661ce02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:31.332049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1048/2120] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:31.335827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a4bda-110f-4f20-9387-52176661ce02 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:31.338450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-626a4bda-110f-4f20-9387-52176661ce02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:31.345466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e3b68-6b1d-452d-a37f-dcffcf911e5b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.356675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a4bda-110f-4f20-9387-52176661ce02 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.357887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:31.359154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.359533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.359912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] PATH_INFO: `/v3/users/6989fa1638584b74994c526b2f7241b4` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.361127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6989fa1638584b74994c526b2f7241b4)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:31.362263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:31.363852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.364023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:31.364187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.364479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] PATH_INFO: `/v3/projects/e060c3513c8c4c459fe9ef1697b77963` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.366273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e060c3513c8c4c459fe9ef1697b77963)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:31.369024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:31.440181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6989fa1638584b74994c526b2f7241b4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.448272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e060c3513c8c4c459fe9ef1697b77963', 'request_id': 'req-04744eb8-c40b-4814-ab65-0f2e3ff64221'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.456009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6989fa1638584b74994c526b2f7241b4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.457187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e060c3513c8c4c459fe9ef1697b77963', 'request_id': 'req-04744eb8-c40b-4814-ab65-0f2e3ff64221'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.468135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e060c3513c8c4c459fe9ef1697b77963', 'request_id': 'req-04744eb8-c40b-4814-ab65-0f2e3ff64221'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.475924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6989fa1638584b74994c526b2f7241b4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.476698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6989fa1638584b74994c526b2f7241b4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.483681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] The token cache is being invalidate because project e060c3513c8c4c459fe9ef1697b77963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:31.484379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.485606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-04744eb8-c40b-4814-ab65-0f2e3ff64221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.490363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1049/2121] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:31 2022] DELETE /identity/v3/projects/e060c3513c8c4c459fe9ef1697b77963 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:31.490363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fbb1a-703a-4019-bff2-9f58f22fb1ec None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.490363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fbb1a-703a-4019-bff2-9f58f22fb1ec None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.490363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fbb1a-703a-4019-bff2-9f58f22fb1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.505375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-0666f6d1-e439-4acc-b821-8a1961375aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6989fa1638584b74994c526b2f7241b4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:31.510364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0e5fbb1a-703a-4019-bff2-9f58f22fb1ec None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:31.511669 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0e5fbb1a-703a-4019-bff2-9f58f22fb1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:31.515278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1073/2122] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:31 2022] DELETE /identity/v3/users/6989fa1638584b74994c526b2f7241b4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:31.515278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba17f2-4308-4580-97db-28828bbe59d4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.515278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba17f2-4308-4580-97db-28828bbe59d4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.515278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba17f2-4308-4580-97db-28828bbe59d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.534958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-60ba17f2-4308-4580-97db-28828bbe59d4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:31.536230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-60ba17f2-4308-4580-97db-28828bbe59d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:31.593668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1050/2123] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:31.595378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a0e3f-5f23-4cb4-8eeb-460f37e0ebce None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.595829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a0e3f-5f23-4cb4-8eeb-460f37e0ebce None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.596113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a0e3f-5f23-4cb4-8eeb-460f37e0ebce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.611916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1074/2124] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:31.613316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba17f2-4308-4580-97db-28828bbe59d4 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:31.626917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-060a0e3f-5f23-4cb4-8eeb-460f37e0ebce None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:31.627107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba17f2-4308-4580-97db-28828bbe59d4 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.628127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-060a0e3f-5f23-4cb4-8eeb-460f37e0ebce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:31.636639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1051/2125] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:31.636775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:31.637692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.637944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12912406-df69-4df0-8046-d33f8a6828d2 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.638198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.638303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12912406-df69-4df0-8046-d33f8a6828d2 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.638585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.638700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12912406-df69-4df0-8046-d33f8a6828d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.639649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:31.640225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:31.642307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:31.667483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-12912406-df69-4df0-8046-d33f8a6828d2 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:31.668915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-12912406-df69-4df0-8046-d33f8a6828d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:31.671190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1075/2126] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:31.671190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:31.675179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:31.680505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1052/2127] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:31.683886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12912406-df69-4df0-8046-d33f8a6828d2 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:31.688314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:31.698575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12912406-df69-4df0-8046-d33f8a6828d2 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.704897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacde7bf-e8b4-4467-8185-0e07fb86b127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:31.705566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacde7bf-e8b4-4467-8185-0e07fb86b127 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.705845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacde7bf-e8b4-4467-8185-0e07fb86b127 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.706121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacde7bf-e8b4-4467-8185-0e07fb86b127 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.706887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacde7bf-e8b4-4467-8185-0e07fb86b127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:31.709290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacde7bf-e8b4-4467-8185-0e07fb86b127 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:31.712896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:31.746678 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1053/2128] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:31 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:31.747839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacde7bf-e8b4-4467-8185-0e07fb86b127 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:31.774508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5403f0-71a3-43d0-8696-0f35a01abe2f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.844536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacde7bf-e8b4-4467-8185-0e07fb86b127 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.859172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400decf8-efe7-41bf-8a42-cceb2947fbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:31.860436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400decf8-efe7-41bf-8a42-cceb2947fbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.860798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400decf8-efe7-41bf-8a42-cceb2947fbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.861203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c833b1-941a-4843-8bd3-b4e9ac4e966a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:31.862075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c833b1-941a-4843-8bd3-b4e9ac4e966a service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.862183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400decf8-efe7-41bf-8a42-cceb2947fbb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.862633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c833b1-941a-4843-8bd3-b4e9ac4e966a service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.863438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400decf8-efe7-41bf-8a42-cceb2947fbb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:31.863829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400decf8-efe7-41bf-8a42-cceb2947fbb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:31.864138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c833b1-941a-4843-8bd3-b4e9ac4e966a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.865909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400decf8-efe7-41bf-8a42-cceb2947fbb7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:31.879219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c833b1-941a-4843-8bd3-b4e9ac4e966a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:31.881490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c833b1-941a-4843-8bd3-b4e9ac4e966a service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:31.882220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1054/2129] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:31.883561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400decf8-efe7-41bf-8a42-cceb2947fbb7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:31.898891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1076/2130] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:31.899359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400decf8-efe7-41bf-8a42-cceb2947fbb7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.900340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c833b1-941a-4843-8bd3-b4e9ac4e966a service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:31.904555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adfa9a-2bcf-4828-85e6-8d98c6b899eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:31.905455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adfa9a-2bcf-4828-85e6-8d98c6b899eb admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.906141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adfa9a-2bcf-4828-85e6-8d98c6b899eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.906496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adfa9a-2bcf-4828-85e6-8d98c6b899eb admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.908765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adfa9a-2bcf-4828-85e6-8d98c6b899eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:31.911616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adfa9a-2bcf-4828-85e6-8d98c6b899eb admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:31.913037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c833b1-941a-4843-8bd3-b4e9ac4e966a service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.918715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f5fd2-155f-457b-b0bf-2ef54417bd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:31.919744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f5fd2-155f-457b-b0bf-2ef54417bd40 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:31.920282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f5fd2-155f-457b-b0bf-2ef54417bd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:31.920801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f5fd2-155f-457b-b0bf-2ef54417bd40 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:31.921747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f5fd2-155f-457b-b0bf-2ef54417bd40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:31.924671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f5fd2-155f-457b-b0bf-2ef54417bd40 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:31.942958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1055/2131] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:31 2022] POST /identity/v3/users => generated 285 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:31.948184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adfa9a-2bcf-4828-85e6-8d98c6b899eb admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:31.963346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adfa9a-2bcf-4828-85e6-8d98c6b899eb admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:31.966343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1077/2132] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:31 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:31.970502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f5fd2-155f-457b-b0bf-2ef54417bd40 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.049566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d4c94-309b-4f08-ac64-f939ce100538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.050432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d4c94-309b-4f08-ac64-f939ce100538 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.050894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d4c94-309b-4f08-ac64-f939ce100538 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.051268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d4c94-309b-4f08-ac64-f939ce100538 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.052031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d4c94-309b-4f08-ac64-f939ce100538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.053655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d4c94-309b-4f08-ac64-f939ce100538 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.059169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f5fd2-155f-457b-b0bf-2ef54417bd40 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.066479 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cf6ab-e019-4c48-b064-370bf289344d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.066660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1056/2133] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:32.067577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cf6ab-e019-4c48-b064-370bf289344d admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.067872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cf6ab-e019-4c48-b064-370bf289344d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.068145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cf6ab-e019-4c48-b064-370bf289344d admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.068915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cf6ab-e019-4c48-b064-370bf289344d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.071127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cf6ab-e019-4c48-b064-370bf289344d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.071740 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d4c94-309b-4f08-ac64-f939ce100538 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.085077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d4c94-309b-4f08-ac64-f939ce100538 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.093730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef6fa4-23ca-41c3-bd15-def36ef47cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.094693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef6fa4-23ca-41c3-bd15-def36ef47cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.095235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef6fa4-23ca-41c3-bd15-def36ef47cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.095671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef6fa4-23ca-41c3-bd15-def36ef47cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.096782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef6fa4-23ca-41c3-bd15-def36ef47cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.100136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef6fa4-23ca-41c3-bd15-def36ef47cf2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.102766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1078/2134] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:31 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:32.112436 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cf6ab-e019-4c48-b064-370bf289344d admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.115316 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1057/2135] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:32.119802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef6fa4-23ca-41c3-bd15-def36ef47cf2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.131164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cf6ab-e019-4c48-b064-370bf289344d admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.136419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef6fa4-23ca-41c3-bd15-def36ef47cf2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.138139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd2e6a-9c49-4c3c-9821-5ff1c606f722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.138851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd2e6a-9c49-4c3c-9821-5ff1c606f722 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.139141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd2e6a-9c49-4c3c-9821-5ff1c606f722 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.140090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd2e6a-9c49-4c3c-9821-5ff1c606f722 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.141523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd2e6a-9c49-4c3c-9821-5ff1c606f722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.143187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec0c21-2b28-4f3c-901e-e8bdc3c1ba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.143395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd2e6a-9c49-4c3c-9821-5ff1c606f722 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.144125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec0c21-2b28-4f3c-901e-e8bdc3c1ba66 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.144500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec0c21-2b28-4f3c-901e-e8bdc3c1ba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.144781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec0c21-2b28-4f3c-901e-e8bdc3c1ba66 admin admin] PATH_INFO: `/v3/projects/d045bf8146b44ac091e7682c54eba872/users/ee25e17d64634ab1b011e8b25541c2b3/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.158832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1079/2136] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:32.159606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd2e6a-9c49-4c3c-9821-5ff1c606f722 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.165619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec0c21-2b28-4f3c-901e-e8bdc3c1ba66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d045bf8146b44ac091e7682c54eba872, user_id=ee25e17d64634ab1b011e8b25541c2b3, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.168507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec0c21-2b28-4f3c-901e-e8bdc3c1ba66 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.172282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd2e6a-9c49-4c3c-9821-5ff1c606f722 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.178570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294784d-d825-4ca7-a05c-e8502196e29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.179328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294784d-d825-4ca7-a05c-e8502196e29d admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.179605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294784d-d825-4ca7-a05c-e8502196e29d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.179879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294784d-d825-4ca7-a05c-e8502196e29d admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.180669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294784d-d825-4ca7-a05c-e8502196e29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.183086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294784d-d825-4ca7-a05c-e8502196e29d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.187991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1058/2137] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:32 2022] PUT /identity/v3/projects/d045bf8146b44ac091e7682c54eba872/users/ee25e17d64634ab1b011e8b25541c2b3/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:32.196229 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1080/2138] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:32.198679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae07cc6-2592-4ad5-9630-31ebb79b6fa5 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.199290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae07cc6-2592-4ad5-9630-31ebb79b6fa5 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.199650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae07cc6-2592-4ad5-9630-31ebb79b6fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.200134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294784d-d825-4ca7-a05c-e8502196e29d admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.228886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294784d-d825-4ca7-a05c-e8502196e29d admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.236092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-bae07cc6-2592-4ad5-9630-31ebb79b6fa5 None None] MFA Rules not processed for user `ee25e17d64634ab1b011e8b25541c2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:32.237932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-bae07cc6-2592-4ad5-9630-31ebb79b6fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:32.240801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:32.251204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:32.339548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1059/2139] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:32.341005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae07cc6-2592-4ad5-9630-31ebb79b6fa5 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.355289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63da179-f4cd-42ce-a774-ada1097c7dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.356739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63da179-f4cd-42ce-a774-ada1097c7dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.357097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63da179-f4cd-42ce-a774-ada1097c7dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.357442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63da179-f4cd-42ce-a774-ada1097c7dc0 admin admin] PATH_INFO: `/v3/projects/718238a5852e4d9a93d85fa8fcf41643/users/5d74c0a687664ac78d3c413251033e49/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.383352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63da179-f4cd-42ce-a774-ada1097c7dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718238a5852e4d9a93d85fa8fcf41643, user_id=5d74c0a687664ac78d3c413251033e49, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.387290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63da179-f4cd-42ce-a774-ada1097c7dc0 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.422913 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1081/2140] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:32 2022] PUT /identity/v3/projects/718238a5852e4d9a93d85fa8fcf41643/users/5d74c0a687664ac78d3c413251033e49/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:32.426511 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bdbdd-7dcb-4689-a7a7-91974489cc4a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.427933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bdbdd-7dcb-4689-a7a7-91974489cc4a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.428201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bdbdd-7dcb-4689-a7a7-91974489cc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.457022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae07cc6-2592-4ad5-9630-31ebb79b6fa5 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.483918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-1d2bdbdd-7dcb-4689-a7a7-91974489cc4a None None] MFA Rules not processed for user `5d74c0a687664ac78d3c413251033e49`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:32.485517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1d2bdbdd-7dcb-4689-a7a7-91974489cc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:32.534107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4614f0-955f-4776-90cf-9c0b064f531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.535134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4614f0-955f-4776-90cf-9c0b064f531b admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.535536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4614f0-955f-4776-90cf-9c0b064f531b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.535908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4614f0-955f-4776-90cf-9c0b064f531b admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.536711 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4614f0-955f-4776-90cf-9c0b064f531b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.539334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4614f0-955f-4776-90cf-9c0b064f531b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.573223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1060/2141] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:32 2022] POST /identity/v3/projects => generated 369 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:32.577476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4614f0-955f-4776-90cf-9c0b064f531b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.604346 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1082/2142] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:32.615311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bdbdd-7dcb-4689-a7a7-91974489cc4a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.617676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1d2bdbdd-7dcb-4689-a7a7-91974489cc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:32.697106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:32.729266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4614f0-955f-4776-90cf-9c0b064f531b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.737653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cdb7d-cdbd-4cef-a43d-0de62373922a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.738498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cdb7d-cdbd-4cef-a43d-0de62373922a admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.738911 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cdb7d-cdbd-4cef-a43d-0de62373922a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.739207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cdb7d-cdbd-4cef-a43d-0de62373922a admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.740092 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cdb7d-cdbd-4cef-a43d-0de62373922a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.743116 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cdb7d-cdbd-4cef-a43d-0de62373922a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.753599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bdbdd-7dcb-4689-a7a7-91974489cc4a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.803298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1061/2143] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:32 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:32.808096 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cdb7d-cdbd-4cef-a43d-0de62373922a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.820067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cdb7d-cdbd-4cef-a43d-0de62373922a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.826368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:32.830512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd8147-e228-49c4-9d75-d88d4684fd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.831368 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd8147-e228-49c4-9d75-d88d4684fd93 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.831667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd8147-e228-49c4-9d75-d88d4684fd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.832023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd8147-e228-49c4-9d75-d88d4684fd93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.832937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd8147-e228-49c4-9d75-d88d4684fd93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.840362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd8147-e228-49c4-9d75-d88d4684fd93 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.855920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4b9ae-bfaa-413f-8042-e438118bda1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.856664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4b9ae-bfaa-413f-8042-e438118bda1e service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.857248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4b9ae-bfaa-413f-8042-e438118bda1e service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.857599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4b9ae-bfaa-413f-8042-e438118bda1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.888825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1062/2144] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:51:32.895497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd8147-e228-49c4-9d75-d88d4684fd93 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:32.899314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-26bd8147-e228-49c4-9d75-d88d4684fd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:32.930353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd8147-e228-49c4-9d75-d88d4684fd93 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:32.935547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:32.936909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:32.937202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:32.937483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] PATH_INFO: `/v3/projects/f025a34ae83e458585973f05e50c5809` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:32.938842 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f025a34ae83e458585973f05e50c5809)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.940752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.980127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4b9ae-bfaa-413f-8042-e438118bda1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:32.981636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4b9ae-bfaa-413f-8042-e438118bda1e service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:32.999743 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1083/2145] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:33.001836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4b9ae-bfaa-413f-8042-e438118bda1e service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.016341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f025a34ae83e458585973f05e50c5809', 'request_id': 'req-39d96637-ef17-4776-b91d-543e8d3a4337'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:33.017260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4b9ae-bfaa-413f-8042-e438118bda1e service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.024915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f025a34ae83e458585973f05e50c5809', 'request_id': 'req-39d96637-ef17-4776-b91d-543e8d3a4337'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:33.038237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f025a34ae83e458585973f05e50c5809', 'request_id': 'req-39d96637-ef17-4776-b91d-543e8d3a4337'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:33.063456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] The token cache is being invalidate because project f025a34ae83e458585973f05e50c5809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:33.063456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:33.064079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-39d96637-ef17-4776-b91d-543e8d3a4337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:33.065787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1063/2146] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:32 2022] DELETE /identity/v3/projects/f025a34ae83e458585973f05e50c5809 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:33.071171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a2235-590b-4679-96a8-35935e6cdcc7 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.071466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a2235-590b-4679-96a8-35935e6cdcc7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.071831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a2235-590b-4679-96a8-35935e6cdcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.096349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e09a2235-590b-4679-96a8-35935e6cdcc7 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:33.097560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e09a2235-590b-4679-96a8-35935e6cdcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:33.146438 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444f9e-9a5c-4dec-ac4f-4975a72b4f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.151107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444f9e-9a5c-4dec-ac4f-4975a72b4f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.151485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444f9e-9a5c-4dec-ac4f-4975a72b4f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.151749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444f9e-9a5c-4dec-ac4f-4975a72b4f68 admin admin] PATH_INFO: `/v3/projects/c073c022326e4a669c7310fe1aae099b/users/97796eca760b4151aed786acea3c51f9/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.176725 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf444f9e-9a5c-4dec-ac4f-4975a72b4f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c073c022326e4a669c7310fe1aae099b, user_id=97796eca760b4151aed786acea3c51f9, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.180849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf444f9e-9a5c-4dec-ac4f-4975a72b4f68 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.187248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1064/2147] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:33.195449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc25edb-aa1f-4446-86cc-077ddfc24d3a None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.196531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc25edb-aa1f-4446-86cc-077ddfc24d3a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.197048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc25edb-aa1f-4446-86cc-077ddfc24d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.205570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1084/2148] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:33 2022] PUT /identity/v3/projects/c073c022326e4a669c7310fe1aae099b/users/97796eca760b4151aed786acea3c51f9/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:33.209512 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444f9e-9a5c-4dec-ac4f-4975a72b4f68 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.211771 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cf444f9e-9a5c-4dec-ac4f-4975a72b4f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:33.233077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-7fc25edb-aa1f-4446-86cc-077ddfc24d3a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:33.237320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7fc25edb-aa1f-4446-86cc-077ddfc24d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:33.319836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:33.327058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444f9e-9a5c-4dec-ac4f-4975a72b4f68 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.334494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53b3c-7a7e-4f7c-ae92-c6f7d85c25e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.335465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53b3c-7a7e-4f7c-ae92-c6f7d85c25e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.336005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53b3c-7a7e-4f7c-ae92-c6f7d85c25e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.336368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53b3c-7a7e-4f7c-ae92-c6f7d85c25e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.337205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f53b3c-7a7e-4f7c-ae92-c6f7d85c25e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.344174 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f53b3c-7a7e-4f7c-ae92-c6f7d85c25e4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.357278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1065/2149] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:33.362657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc25edb-aa1f-4446-86cc-077ddfc24d3a None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.370258 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1085/2150] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:33.375248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53b3c-7a7e-4f7c-ae92-c6f7d85c25e4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.378326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc25edb-aa1f-4446-86cc-077ddfc24d3a None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.383776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93504a2d-f769-4464-aea6-92c824cbf546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.384538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93504a2d-f769-4464-aea6-92c824cbf546 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.385037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93504a2d-f769-4464-aea6-92c824cbf546 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.385379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93504a2d-f769-4464-aea6-92c824cbf546 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.386197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93504a2d-f769-4464-aea6-92c824cbf546 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:33.386587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93504a2d-f769-4464-aea6-92c824cbf546 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.387618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53b3c-7a7e-4f7c-ae92-c6f7d85c25e4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.388421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93504a2d-f769-4464-aea6-92c824cbf546 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.393136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d606f-d4cf-4c28-88d7-92bd43973cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.394134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d606f-d4cf-4c28-88d7-92bd43973cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.394712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d606f-d4cf-4c28-88d7-92bd43973cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.395118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d606f-d4cf-4c28-88d7-92bd43973cd3 admin admin] PATH_INFO: `/v3/projects/c073c022326e4a669c7310fe1aae099b/users/97796eca760b4151aed786acea3c51f9/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.397398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d606f-d4cf-4c28-88d7-92bd43973cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c073c022326e4a669c7310fe1aae099b, user_id=97796eca760b4151aed786acea3c51f9, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.400962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d606f-d4cf-4c28-88d7-92bd43973cd3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.402982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1066/2151] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:33.407743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93504a2d-f769-4464-aea6-92c824cbf546 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.419851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1086/2152] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:33 2022] PUT /identity/v3/projects/c073c022326e4a669c7310fe1aae099b/users/97796eca760b4151aed786acea3c51f9/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:33.420076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93504a2d-f769-4464-aea6-92c824cbf546 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.429986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6869b-4000-4bbc-8afe-aeb5cbc7cc4a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.430372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6869b-4000-4bbc-8afe-aeb5cbc7cc4a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.430682 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6869b-4000-4bbc-8afe-aeb5cbc7cc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.447268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:33.495614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-31b6869b-4000-4bbc-8afe-aeb5cbc7cc4a None None] MFA Rules not processed for user `97796eca760b4151aed786acea3c51f9`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:33.497352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-31b6869b-4000-4bbc-8afe-aeb5cbc7cc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:33.555050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af23253-d80d-4323-98b6-97c7e77b2a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.555823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af23253-d80d-4323-98b6-97c7e77b2a28 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.556128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af23253-d80d-4323-98b6-97c7e77b2a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.556412 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af23253-d80d-4323-98b6-97c7e77b2a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.557123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af23253-d80d-4323-98b6-97c7e77b2a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.559334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af23253-d80d-4323-98b6-97c7e77b2a28 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.584950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1087/2153] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:33.603456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1067/2154] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:33 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:33.603651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6869b-4000-4bbc-8afe-aeb5cbc7cc4a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.603651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-31b6869b-4000-4bbc-8afe-aeb5cbc7cc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:33.604991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af23253-d80d-4323-98b6-97c7e77b2a28 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.668542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6869b-4000-4bbc-8afe-aeb5cbc7cc4a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.699535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af23253-d80d-4323-98b6-97c7e77b2a28 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.704977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93195e6-d5ad-48de-8268-843526064dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.705668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93195e6-d5ad-48de-8268-843526064dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.705954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93195e6-d5ad-48de-8268-843526064dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.706234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93195e6-d5ad-48de-8268-843526064dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.706979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93195e6-d5ad-48de-8268-843526064dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.709042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93195e6-d5ad-48de-8268-843526064dd6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.733809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecabee-6de4-4330-bd0d-7a714e0a413f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.735079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecabee-6de4-4330-bd0d-7a714e0a413f service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.735364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecabee-6de4-4330-bd0d-7a714e0a413f service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.735642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecabee-6de4-4330-bd0d-7a714e0a413f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.736208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1068/2155] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:33 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:33.740467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93195e6-d5ad-48de-8268-843526064dd6 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.754331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93195e6-d5ad-48de-8268-843526064dd6 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.761284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059da7d6-fa8d-4e7e-bc12-961977f26651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.762128 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059da7d6-fa8d-4e7e-bc12-961977f26651 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.762485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059da7d6-fa8d-4e7e-bc12-961977f26651 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.762873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059da7d6-fa8d-4e7e-bc12-961977f26651 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.763813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059da7d6-fa8d-4e7e-bc12-961977f26651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.766233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059da7d6-fa8d-4e7e-bc12-961977f26651 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.779068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1069/2156] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:33.782605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059da7d6-fa8d-4e7e-bc12-961977f26651 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.795595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059da7d6-fa8d-4e7e-bc12-961977f26651 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.802600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d132d-fc7a-493f-b198-028bf6e17d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.803348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d132d-fc7a-493f-b198-028bf6e17d4f admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.803760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d132d-fc7a-493f-b198-028bf6e17d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.804090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d132d-fc7a-493f-b198-028bf6e17d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.804767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d132d-fc7a-493f-b198-028bf6e17d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.807226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d132d-fc7a-493f-b198-028bf6e17d4f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.822187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1070/2157] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:33.826472 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d132d-fc7a-493f-b198-028bf6e17d4f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.839445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d132d-fc7a-493f-b198-028bf6e17d4f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:33.862640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:33.863281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff2e21-6c29-438e-ab61-b37c15e5a306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:33.864368 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff2e21-6c29-438e-ab61-b37c15e5a306 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.864837 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff2e21-6c29-438e-ab61-b37c15e5a306 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.865237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff2e21-6c29-438e-ab61-b37c15e5a306 admin admin] PATH_INFO: `/v3/projects/bf53255e905344409fd64e4fba710dbc/users/e872bc2e7b214f0889ba95cc9352c59b/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.874845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecabee-6de4-4330-bd0d-7a714e0a413f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.876322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecabee-6de4-4330-bd0d-7a714e0a413f service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.893073 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff2e21-6c29-438e-ab61-b37c15e5a306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf53255e905344409fd64e4fba710dbc, user_id=e872bc2e7b214f0889ba95cc9352c59b, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:33.895780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff2e21-6c29-438e-ab61-b37c15e5a306 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:33.914204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1071/2158] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:33 2022] PUT /identity/v3/projects/bf53255e905344409fd64e4fba710dbc/users/e872bc2e7b214f0889ba95cc9352c59b/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:33.914655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1088/2159] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:33 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:51:33.919740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adf5a8-70e2-419f-9113-5537920d308c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:33.920259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adf5a8-70e2-419f-9113-5537920d308c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:33.920831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adf5a8-70e2-419f-9113-5537920d308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:33.945281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-09adf5a8-70e2-419f-9113-5537920d308c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:33.946604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-09adf5a8-70e2-419f-9113-5537920d308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:33.964250 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff2e21-6c29-438e-ab61-b37c15e5a306 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:33.966080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b9ff2e21-6c29-438e-ab61-b37c15e5a306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:33.973463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:34.046772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1089/2160] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:34.053490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff2e21-6c29-438e-ab61-b37c15e5a306 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:34.056004 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adf5a8-70e2-419f-9113-5537920d308c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:34.057581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-09adf5a8-70e2-419f-9113-5537920d308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:34.058319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab88df-c7e3-4bd1-967f-445e26a6453c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:34.059219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab88df-c7e3-4bd1-967f-445e26a6453c service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:34.059662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab88df-c7e3-4bd1-967f-445e26a6453c service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:34.060078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab88df-c7e3-4bd1-967f-445e26a6453c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:34.063680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0eab88df-c7e3-4bd1-967f-445e26a6453c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:34.123116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adf5a8-70e2-419f-9113-5537920d308c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:34.127458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3cec9-fe26-4537-82fe-17f8cbd6c2b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:34.128125 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3cec9-fe26-4537-82fe-17f8cbd6c2b9 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:34.128407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3cec9-fe26-4537-82fe-17f8cbd6c2b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:34.128800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3cec9-fe26-4537-82fe-17f8cbd6c2b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:34.139498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3cec9-fe26-4537-82fe-17f8cbd6c2b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:34.141022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3cec9-fe26-4537-82fe-17f8cbd6c2b9 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:34.151755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1090/2161] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:34.170005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab88df-c7e3-4bd1-967f-445e26a6453c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:34.172002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab88df-c7e3-4bd1-967f-445e26a6453c service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:34.192436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1072/2162] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:35.621331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ea04-7d45-460d-a815-909e028f6aee None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:35.621855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ea04-7d45-460d-a815-909e028f6aee None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:35.621947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ea04-7d45-460d-a815-909e028f6aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:35.643102 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d7cb5-dfa9-4b04-ab7e-6fcbd8b99336 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:35.643596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d7cb5-dfa9-4b04-ab7e-6fcbd8b99336 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:35.643688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d7cb5-dfa9-4b04-ab7e-6fcbd8b99336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:35.656002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4ec6ea04-7d45-460d-a815-909e028f6aee None None] MFA Rules not processed for user `5d74c0a687664ac78d3c413251033e49`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:35.658847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4ec6ea04-7d45-460d-a815-909e028f6aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:35.718332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b62d7cb5-dfa9-4b04-ab7e-6fcbd8b99336 None None] MFA Rules not processed for user `e872bc2e7b214f0889ba95cc9352c59b`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:35.722260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b62d7cb5-dfa9-4b04-ab7e-6fcbd8b99336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:35.736898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1091/2163] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:35.934880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1073/2164] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:35.969526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ea04-7d45-460d-a815-909e028f6aee None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:35.987658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ea04-7d45-460d-a815-909e028f6aee None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:35.997071 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1aa9e-72a4-4b94-9d73-6af2b0368101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:35.998102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1aa9e-72a4-4b94-9d73-6af2b0368101 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:35.998643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1aa9e-72a4-4b94-9d73-6af2b0368101 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:36.001486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1aa9e-72a4-4b94-9d73-6af2b0368101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:36.014925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1aa9e-72a4-4b94-9d73-6af2b0368101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:36.016537 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1aa9e-72a4-4b94-9d73-6af2b0368101 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:36.032459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1092/2165] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:37.068820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4c591-4900-4d96-b25b-c8e4dca71224 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:37.069573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4c591-4900-4d96-b25b-c8e4dca71224 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:37.069922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4c591-4900-4d96-b25b-c8e4dca71224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:37.106467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-36f4c591-4900-4d96-b25b-c8e4dca71224 None None] MFA Rules not processed for user `f8fcb09990db433e8ac0af3de37bdbc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:37.107745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-36f4c591-4900-4d96-b25b-c8e4dca71224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:37.157519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:37.198871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1074/2166] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:37.210061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a247-c4cc-4642-b001-35df33311684 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:37.210524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a247-c4cc-4642-b001-35df33311684 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:37.210524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a247-c4cc-4642-b001-35df33311684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:37.231571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e3a4a247-c4cc-4642-b001-35df33311684 None None] MFA Rules not processed for user `f8fcb09990db433e8ac0af3de37bdbc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:37.232912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e3a4a247-c4cc-4642-b001-35df33311684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:37.243456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1093/2167] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:37.255189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4c591-4900-4d96-b25b-c8e4dca71224 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:37.257864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-36f4c591-4900-4d96-b25b-c8e4dca71224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:37.337994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4c591-4900-4d96-b25b-c8e4dca71224 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:37.343864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aff550-2fdd-4d66-8cb5-bb6c9b69590d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:37.344512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aff550-2fdd-4d66-8cb5-bb6c9b69590d service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:37.344793 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aff550-2fdd-4d66-8cb5-bb6c9b69590d service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:37.345082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aff550-2fdd-4d66-8cb5-bb6c9b69590d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:37.353067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aff550-2fdd-4d66-8cb5-bb6c9b69590d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:37.354637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aff550-2fdd-4d66-8cb5-bb6c9b69590d service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:37.363857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1075/2168] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:38.247445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee3a39-0205-4706-bfcd-067b025239e2 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:38.248390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee3a39-0205-4706-bfcd-067b025239e2 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:38.248895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee3a39-0205-4706-bfcd-067b025239e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:38.283250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b6ee3a39-0205-4706-bfcd-067b025239e2 None None] MFA Rules not processed for user `f8fcb09990db433e8ac0af3de37bdbc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:38.288198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b6ee3a39-0205-4706-bfcd-067b025239e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:38.296921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1094/2169] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:51:38.636975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aff550-2fdd-4d66-8cb5-bb6c9b69590d service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:38.650433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aff550-2fdd-4d66-8cb5-bb6c9b69590d service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:38.656808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:38.657976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:38.658508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:38.659129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] PATH_INFO: `/v3/users/ee25e17d64634ab1b011e8b25541c2b3` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:38.660805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee25e17d64634ab1b011e8b25541c2b3)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:38.663364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:38.784417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee25e17d64634ab1b011e8b25541c2b3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:38.787674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d6dbc-f683-4c4f-8f95-311fa0cd3e78 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:38.788157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d6dbc-f683-4c4f-8f95-311fa0cd3e78 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:38.788603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d6dbc-f683-4c4f-8f95-311fa0cd3e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:38.813740 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee25e17d64634ab1b011e8b25541c2b3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:38.818842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-581d6dbc-f683-4c4f-8f95-311fa0cd3e78 None None] MFA Rules not processed for user `e872bc2e7b214f0889ba95cc9352c59b`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:38.831043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee25e17d64634ab1b011e8b25541c2b3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:38.834003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-581d6dbc-f683-4c4f-8f95-311fa0cd3e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:38.837150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee25e17d64634ab1b011e8b25541c2b3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:38.838452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee25e17d64634ab1b011e8b25541c2b3'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:38.856216 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1095/2170] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:38.896583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1076/2171] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:38 2022] DELETE /identity/v3/users/ee25e17d64634ab1b011e8b25541c2b3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:38.900187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd044-c77b-4cfc-8cf1-514f4b5beef1 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:38.901112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd044-c77b-4cfc-8cf1-514f4b5beef1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:38.901902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd044-c77b-4cfc-8cf1-514f4b5beef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:38.968167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-229dd044-c77b-4cfc-8cf1-514f4b5beef1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:38.970356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-229dd044-c77b-4cfc-8cf1-514f4b5beef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:38.983111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:39.086514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1096/2172] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:39.097348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:39.098906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:39.155814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:39.182928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affde1c5-93db-4020-8c7c-2e49656d5cb9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:39.209415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c5603-2c64-45ad-93dd-4e422b4e2670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:39.210598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c5603-2c64-45ad-93dd-4e422b4e2670 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:39.211227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c5603-2c64-45ad-93dd-4e422b4e2670 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:39.213005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c5603-2c64-45ad-93dd-4e422b4e2670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:39.225660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c5603-2c64-45ad-93dd-4e422b4e2670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:39.228115 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c5603-2c64-45ad-93dd-4e422b4e2670 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:39.257711 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1077/2173] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:39.944053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dd044-c77b-4cfc-8cf1-514f4b5beef1 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:39.946262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-229dd044-c77b-4cfc-8cf1-514f4b5beef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:39.963476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dd044-c77b-4cfc-8cf1-514f4b5beef1 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:39.970818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:39.972734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:39.973219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:39.973755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] PATH_INFO: `/v3/projects/d045bf8146b44ac091e7682c54eba872` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:39.975610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d045bf8146b44ac091e7682c54eba872)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:39.977783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:40.076470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd045bf8146b44ac091e7682c54eba872', 'request_id': 'req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.095732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd045bf8146b44ac091e7682c54eba872', 'request_id': 'req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.112858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd045bf8146b44ac091e7682c54eba872', 'request_id': 'req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.151944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] The token cache is being invalidate because project d045bf8146b44ac091e7682c54eba872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:40.152254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.157631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.161700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1097/2174] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:39 2022] DELETE /identity/v3/projects/d045bf8146b44ac091e7682c54eba872 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:40.166048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c5603-2c64-45ad-93dd-4e422b4e2670 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:40.198282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c5603-2c64-45ad-93dd-4e422b4e2670 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:40.400467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:40.401435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:40.407106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:40.408193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] PATH_INFO: `/v3/users/97796eca760b4151aed786acea3c51f9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:40.413065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97796eca760b4151aed786acea3c51f9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:40.423083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:40.597405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97796eca760b4151aed786acea3c51f9'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.622362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97796eca760b4151aed786acea3c51f9'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.632782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97796eca760b4151aed786acea3c51f9'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.636402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97796eca760b4151aed786acea3c51f9'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.637335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cb4a8d13-09c4-4def-9e50-0a950ec03019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97796eca760b4151aed786acea3c51f9'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:40.692881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1078/2175] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:40 2022] DELETE /identity/v3/users/97796eca760b4151aed786acea3c51f9 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:41.612455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:41.614299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:41.653024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757142c-b8d6-4382-b6d1-f3eb40fb36ed admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:41.762386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:41.763629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:41.764089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:41.764559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] PATH_INFO: `/v3/projects/c073c022326e4a669c7310fe1aae099b` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:41.766238 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] RBAC: Authorizing `identity:delete_project(project_id=c073c022326e4a669c7310fe1aae099b)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:41.794643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:41.939924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:41.970597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c073c022326e4a669c7310fe1aae099b', 'request_id': 'req-f45414b6-0931-428f-b469-71e191ce41af'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:42.018871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c073c022326e4a669c7310fe1aae099b', 'request_id': 'req-f45414b6-0931-428f-b469-71e191ce41af'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:42.057040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c073c022326e4a669c7310fe1aae099b', 'request_id': 'req-f45414b6-0931-428f-b469-71e191ce41af'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:42.078428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] The token cache is being invalidate because project c073c022326e4a669c7310fe1aae099b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:42.078723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:42.080345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f45414b6-0931-428f-b469-71e191ce41af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:42.084415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1098/2176] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:41 2022] DELETE /identity/v3/projects/c073c022326e4a669c7310fe1aae099b => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:42.088997 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e46b7b-b38f-4620-a048-c363c2c9e1c6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:42.089998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e46b7b-b38f-4620-a048-c363c2c9e1c6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:42.090434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e46b7b-b38f-4620-a048-c363c2c9e1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:42.142323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-61e46b7b-b38f-4620-a048-c363c2c9e1c6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:42.143598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-61e46b7b-b38f-4620-a048-c363c2c9e1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:42.295993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:42.351115 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1079/2177] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:42.359233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3093e-dc73-4fd9-a190-30eaad6d6d99 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:42.359686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3093e-dc73-4fd9-a190-30eaad6d6d99 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:42.360091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3093e-dc73-4fd9-a190-30eaad6d6d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:42.399784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-c2c3093e-dc73-4fd9-a190-30eaad6d6d99 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:42.405006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c2c3093e-dc73-4fd9-a190-30eaad6d6d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:42.432382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1099/2178] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:42.437111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e46b7b-b38f-4620-a048-c363c2c9e1c6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:42.455848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e46b7b-b38f-4620-a048-c363c2c9e1c6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:42.464925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d58628-0df9-4bf1-803d-395a64d75fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:42.466083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d58628-0df9-4bf1-803d-395a64d75fac admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:42.466546 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d58628-0df9-4bf1-803d-395a64d75fac admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:42.467001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d58628-0df9-4bf1-803d-395a64d75fac admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:42.467999 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d58628-0df9-4bf1-803d-395a64d75fac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:42.468462 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d58628-0df9-4bf1-803d-395a64d75fac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:42.470325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d58628-0df9-4bf1-803d-395a64d75fac admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:42.497975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1080/2179] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:42.503430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3093e-dc73-4fd9-a190-30eaad6d6d99 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:42.514057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3093e-dc73-4fd9-a190-30eaad6d6d99 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:42.520036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a497d37-7b28-4064-a530-eb4e14ac26d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:42.521125 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a497d37-7b28-4064-a530-eb4e14ac26d0 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:42.521559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a497d37-7b28-4064-a530-eb4e14ac26d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:42.521986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a497d37-7b28-4064-a530-eb4e14ac26d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:42.522988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a497d37-7b28-4064-a530-eb4e14ac26d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:42.525266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a497d37-7b28-4064-a530-eb4e14ac26d0 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:42.580042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1100/2180] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:42 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:42.580227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d58628-0df9-4bf1-803d-395a64d75fac admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:42.595327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d58628-0df9-4bf1-803d-395a64d75fac admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:42.679648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861886f6-ccc8-4492-8300-9743e1e91943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:42.680657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861886f6-ccc8-4492-8300-9743e1e91943 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:42.681476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861886f6-ccc8-4492-8300-9743e1e91943 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:42.681874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861886f6-ccc8-4492-8300-9743e1e91943 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:42.682805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861886f6-ccc8-4492-8300-9743e1e91943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:42.690185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861886f6-ccc8-4492-8300-9743e1e91943 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:42.752604 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1081/2181] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:42 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:42.753915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a497d37-7b28-4064-a530-eb4e14ac26d0 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:42.770322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a497d37-7b28-4064-a530-eb4e14ac26d0 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:42.787407 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefa880-2ff6-4dc8-85da-652fcb4dec16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:42.788108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefa880-2ff6-4dc8-85da-652fcb4dec16 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:42.788418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefa880-2ff6-4dc8-85da-652fcb4dec16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:42.788692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefa880-2ff6-4dc8-85da-652fcb4dec16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:42.789621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefa880-2ff6-4dc8-85da-652fcb4dec16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:42.803291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefa880-2ff6-4dc8-85da-652fcb4dec16 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:42.832220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1101/2182] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:42.837239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861886f6-ccc8-4492-8300-9743e1e91943 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:42.857822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861886f6-ccc8-4492-8300-9743e1e91943 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:42.865215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8812aad-03f5-4aa1-8bab-3fcc5bba5097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:42.865926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8812aad-03f5-4aa1-8bab-3fcc5bba5097 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:42.866212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8812aad-03f5-4aa1-8bab-3fcc5bba5097 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:42.866495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8812aad-03f5-4aa1-8bab-3fcc5bba5097 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:42.868759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8812aad-03f5-4aa1-8bab-3fcc5bba5097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:42.869056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8812aad-03f5-4aa1-8bab-3fcc5bba5097 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:42.908788 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1082/2183] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:42.909563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefa880-2ff6-4dc8-85da-652fcb4dec16 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:42.928392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefa880-2ff6-4dc8-85da-652fcb4dec16 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:42.939932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad146d9-f933-4286-b9c5-44b602a61861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:42.940997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad146d9-f933-4286-b9c5-44b602a61861 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:42.942011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad146d9-f933-4286-b9c5-44b602a61861 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:42.943462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad146d9-f933-4286-b9c5-44b602a61861 admin admin] PATH_INFO: `/v3/projects/f633026c819145d5adc0e67721b60ac9/users/4211835ed7434f57b3122d2831afee77/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:42.972860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad146d9-f933-4286-b9c5-44b602a61861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f633026c819145d5adc0e67721b60ac9, user_id=4211835ed7434f57b3122d2831afee77, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:42.975637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad146d9-f933-4286-b9c5-44b602a61861 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:42.998000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1102/2184] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:42 2022] PUT /identity/v3/projects/f633026c819145d5adc0e67721b60ac9/users/4211835ed7434f57b3122d2831afee77/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:43.009247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d38654-fc40-4284-8faa-83652f786345 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:43.009558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d38654-fc40-4284-8faa-83652f786345 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:43.009836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d38654-fc40-4284-8faa-83652f786345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:43.064913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-63d38654-fc40-4284-8faa-83652f786345 None None] MFA Rules not processed for user `4211835ed7434f57b3122d2831afee77`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:43.066487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-63d38654-fc40-4284-8faa-83652f786345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:43.179163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1083/2185] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:43.183814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad146d9-f933-4286-b9c5-44b602a61861 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:43.260226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad146d9-f933-4286-b9c5-44b602a61861 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:43.265072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebc67b-b652-4527-ad3b-752eb20dde04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:43.265734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebc67b-b652-4527-ad3b-752eb20dde04 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:43.266017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebc67b-b652-4527-ad3b-752eb20dde04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:43.266299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebc67b-b652-4527-ad3b-752eb20dde04 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:43.267026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebc67b-b652-4527-ad3b-752eb20dde04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:43.268939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebc67b-b652-4527-ad3b-752eb20dde04 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:43.306391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1103/2186] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:43 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:43.310414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d38654-fc40-4284-8faa-83652f786345 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:43.321752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d38654-fc40-4284-8faa-83652f786345 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:43.370998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:43.384777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46a31-6cbd-4b57-b322-bfb4e1a475e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:43.385738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46a31-6cbd-4b57-b322-bfb4e1a475e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:43.386023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46a31-6cbd-4b57-b322-bfb4e1a475e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:43.386303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46a31-6cbd-4b57-b322-bfb4e1a475e4 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:43.387046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae46a31-6cbd-4b57-b322-bfb4e1a475e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:43.389003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae46a31-6cbd-4b57-b322-bfb4e1a475e4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:43.417731 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1084/2187] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:43 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:43.418918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebc67b-b652-4527-ad3b-752eb20dde04 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:43.430405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebc67b-b652-4527-ad3b-752eb20dde04 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:43.453510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af3c66-736b-47a2-8c4f-a53af370446a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:43.453901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af3c66-736b-47a2-8c4f-a53af370446a admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:43.454094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af3c66-736b-47a2-8c4f-a53af370446a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:43.454564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af3c66-736b-47a2-8c4f-a53af370446a admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:43.455410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af3c66-736b-47a2-8c4f-a53af370446a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:43.457233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af3c66-736b-47a2-8c4f-a53af370446a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:43.474895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1104/2188] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:43.476207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46a31-6cbd-4b57-b322-bfb4e1a475e4 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:43.487765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46a31-6cbd-4b57-b322-bfb4e1a475e4 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:43.492462 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5febb-be42-4dc2-9aa7-53978dfb4817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:43.493603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5febb-be42-4dc2-9aa7-53978dfb4817 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:43.494301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5febb-be42-4dc2-9aa7-53978dfb4817 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:43.494301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5febb-be42-4dc2-9aa7-53978dfb4817 admin admin] PATH_INFO: `/v3/projects/6627bad4cb6049a382bf15c9bbc9a10e/users/b732089118834cdf9181d388cacfc191/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:43.512821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5febb-be42-4dc2-9aa7-53978dfb4817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6627bad4cb6049a382bf15c9bbc9a10e, user_id=b732089118834cdf9181d388cacfc191, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:43.516160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5febb-be42-4dc2-9aa7-53978dfb4817 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:43.533415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1085/2189] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:43 2022] PUT /identity/v3/projects/6627bad4cb6049a382bf15c9bbc9a10e/users/b732089118834cdf9181d388cacfc191/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:43.542400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af3c66-736b-47a2-8c4f-a53af370446a admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:43.632845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af3c66-736b-47a2-8c4f-a53af370446a admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:43.683767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:43.729084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2b3fb-3760-4169-a6af-b1be60285e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:43.731040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2b3fb-3760-4169-a6af-b1be60285e2a admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:43.731415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2b3fb-3760-4169-a6af-b1be60285e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:43.731884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2b3fb-3760-4169-a6af-b1be60285e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:43.733068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2b3fb-3760-4169-a6af-b1be60285e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:43.735901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2b3fb-3760-4169-a6af-b1be60285e2a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:43.752676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5febb-be42-4dc2-9aa7-53978dfb4817 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:43.762904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1105/2190] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:43.765336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5febb-be42-4dc2-9aa7-53978dfb4817 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:43.773173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776f13d-6a0c-4911-b959-595d563f7916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:43.774513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776f13d-6a0c-4911-b959-595d563f7916 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:43.775313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776f13d-6a0c-4911-b959-595d563f7916 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:43.776387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776f13d-6a0c-4911-b959-595d563f7916 admin admin] PATH_INFO: `/v3/projects/6627bad4cb6049a382bf15c9bbc9a10e/users/b732089118834cdf9181d388cacfc191/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:43.778987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776f13d-6a0c-4911-b959-595d563f7916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6627bad4cb6049a382bf15c9bbc9a10e, user_id=b732089118834cdf9181d388cacfc191, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:43.782535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776f13d-6a0c-4911-b959-595d563f7916 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:43.807406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9881e5-2f1e-46e9-9220-b4094522c763 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:43.807836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9881e5-2f1e-46e9-9220-b4094522c763 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:43.808348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9881e5-2f1e-46e9-9220-b4094522c763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:43.810482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1086/2191] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:43 2022] PUT /identity/v3/projects/6627bad4cb6049a382bf15c9bbc9a10e/users/b732089118834cdf9181d388cacfc191/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:43.841419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-0f9881e5-2f1e-46e9-9220-b4094522c763 None None] MFA Rules not processed for user `b732089118834cdf9181d388cacfc191`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:43.842906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0f9881e5-2f1e-46e9-9220-b4094522c763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:43.929501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1106/2192] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:43.938766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776f13d-6a0c-4911-b959-595d563f7916 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:43.940598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a776f13d-6a0c-4911-b959-595d563f7916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:44.000566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776f13d-6a0c-4911-b959-595d563f7916 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:44.004341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af82d7-b2db-41ae-af53-85753f11f959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:44.005204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af82d7-b2db-41ae-af53-85753f11f959 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:44.005565 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af82d7-b2db-41ae-af53-85753f11f959 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:44.005917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af82d7-b2db-41ae-af53-85753f11f959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:44.016815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af82d7-b2db-41ae-af53-85753f11f959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:44.018263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af82d7-b2db-41ae-af53-85753f11f959 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:44.030361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1087/2193] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:44.060419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9881e5-2f1e-46e9-9220-b4094522c763 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:44.126012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9881e5-2f1e-46e9-9220-b4094522c763 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:44.130673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:44.131594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:44.131876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:44.132766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] PATH_INFO: `/v3/users/4211835ed7434f57b3122d2831afee77` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:44.133596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4211835ed7434f57b3122d2831afee77)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:44.135613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:44.191925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4211835ed7434f57b3122d2831afee77'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.202714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4211835ed7434f57b3122d2831afee77'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.205901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:44.214284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4211835ed7434f57b3122d2831afee77'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.215806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4211835ed7434f57b3122d2831afee77'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.238242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4211835ed7434f57b3122d2831afee77'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.250651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1107/2194] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:44 2022] DELETE /identity/v3/users/4211835ed7434f57b3122d2831afee77 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:44.252810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8611d8-fce1-4073-b361-17914e6782aa None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:44.253288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8611d8-fce1-4073-b361-17914e6782aa None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:44.253616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8611d8-fce1-4073-b361-17914e6782aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:44.281095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-2d8611d8-fce1-4073-b361-17914e6782aa None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:44.282650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2d8611d8-fce1-4073-b361-17914e6782aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:44.390583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1088/2195] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:44.403684 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:44.405742 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:44.496886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1928da-0617-4a3e-90ad-e98035f51efb admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:44.501193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149ea15-0f1a-42c9-93d7-20e7f346d278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:44.501916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149ea15-0f1a-42c9-93d7-20e7f346d278 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:44.502267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149ea15-0f1a-42c9-93d7-20e7f346d278 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:44.502618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149ea15-0f1a-42c9-93d7-20e7f346d278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:44.511995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149ea15-0f1a-42c9-93d7-20e7f346d278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:44.513659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149ea15-0f1a-42c9-93d7-20e7f346d278 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:44.523477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1108/2196] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:44.800039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8611d8-fce1-4073-b361-17914e6782aa None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:44.801081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2d8611d8-fce1-4073-b361-17914e6782aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:44.815469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8611d8-fce1-4073-b361-17914e6782aa None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:44.821213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:44.821998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:44.822287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:44.822567 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] PATH_INFO: `/v3/projects/f633026c819145d5adc0e67721b60ac9` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:44.823971 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f633026c819145d5adc0e67721b60ac9)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:44.825873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:44.867672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:44.874507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f633026c819145d5adc0e67721b60ac9', 'request_id': 'req-eaffbf00-4ad5-4a58-a981-eb45630e4b21'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.889569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f633026c819145d5adc0e67721b60ac9', 'request_id': 'req-eaffbf00-4ad5-4a58-a981-eb45630e4b21'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.897154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f633026c819145d5adc0e67721b60ac9', 'request_id': 'req-eaffbf00-4ad5-4a58-a981-eb45630e4b21'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.908455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] The token cache is being invalidate because project f633026c819145d5adc0e67721b60ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:44.908740 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.909353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:44.910912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1089/2197] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:44 2022] DELETE /identity/v3/projects/f633026c819145d5adc0e67721b60ac9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:44.915137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149ea15-0f1a-42c9-93d7-20e7f346d278 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:44.925925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149ea15-0f1a-42c9-93d7-20e7f346d278 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:44.986151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:44.987051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:44.987333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:44.987610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] PATH_INFO: `/v3/users/b732089118834cdf9181d388cacfc191` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:44.988803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b732089118834cdf9181d388cacfc191)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:44.990716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:45.044256 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b732089118834cdf9181d388cacfc191'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.052672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b732089118834cdf9181d388cacfc191'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.059792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b732089118834cdf9181d388cacfc191'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.060477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b732089118834cdf9181d388cacfc191'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.071273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:45.072791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-a334c045-929e-416d-83e1-3685e16607f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b732089118834cdf9181d388cacfc191'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.082959 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1109/2198] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:44 2022] DELETE /identity/v3/users/b732089118834cdf9181d388cacfc191 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:45.355636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:45.357300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:45.370289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffbf00-4ad5-4a58-a981-eb45630e4b21 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:45.428354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:45.429214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.429497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.429775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] PATH_INFO: `/v3/projects/6627bad4cb6049a382bf15c9bbc9a10e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.431002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6627bad4cb6049a382bf15c9bbc9a10e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:45.432915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:45.479254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6627bad4cb6049a382bf15c9bbc9a10e', 'request_id': 'req-c185051c-7192-4296-9be2-2bacc368af41'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.485498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6627bad4cb6049a382bf15c9bbc9a10e', 'request_id': 'req-c185051c-7192-4296-9be2-2bacc368af41'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.493548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6627bad4cb6049a382bf15c9bbc9a10e', 'request_id': 'req-c185051c-7192-4296-9be2-2bacc368af41'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.505084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] The token cache is being invalidate because project 6627bad4cb6049a382bf15c9bbc9a10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:45.505376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.506018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c185051c-7192-4296-9be2-2bacc368af41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:45.507594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1090/2199] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:45 2022] DELETE /identity/v3/projects/6627bad4cb6049a382bf15c9bbc9a10e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:45.512656 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08ae69-e608-4d78-95d2-533fb8468a69 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.512950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08ae69-e608-4d78-95d2-533fb8468a69 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.513237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08ae69-e608-4d78-95d2-533fb8468a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.531902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-eb08ae69-e608-4d78-95d2-533fb8468a69 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:45.533178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-eb08ae69-e608-4d78-95d2-533fb8468a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:45.600698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1110/2200] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:45.608721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb1a5f-c2b7-43d4-aa03-a240a098b171 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.609145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb1a5f-c2b7-43d4-aa03-a240a098b171 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.609707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb1a5f-c2b7-43d4-aa03-a240a098b171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.633048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d6fb1a5f-c2b7-43d4-aa03-a240a098b171 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:45.634267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d6fb1a5f-c2b7-43d4-aa03-a240a098b171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:45.643903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1091/2201] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:45.652200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08ae69-e608-4d78-95d2-533fb8468a69 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:45.662328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08ae69-e608-4d78-95d2-533fb8468a69 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:45.666938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3b134-5fe8-4776-8f71-5d89b023b68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:45.667602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3b134-5fe8-4776-8f71-5d89b023b68a admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.667884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3b134-5fe8-4776-8f71-5d89b023b68a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.668164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3b134-5fe8-4776-8f71-5d89b023b68a admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.668924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3b134-5fe8-4776-8f71-5d89b023b68a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:45.669235 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3b134-5fe8-4776-8f71-5d89b023b68a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:45.670900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3b134-5fe8-4776-8f71-5d89b023b68a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:45.683045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1111/2202] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:45.687744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb1a5f-c2b7-43d4-aa03-a240a098b171 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:45.698384 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb1a5f-c2b7-43d4-aa03-a240a098b171 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:45.707319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc905da-d83d-4a2c-a727-dc2e1b766e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:45.708477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc905da-d83d-4a2c-a727-dc2e1b766e6c admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.709001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc905da-d83d-4a2c-a727-dc2e1b766e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.709512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc905da-d83d-4a2c-a727-dc2e1b766e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.710274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc905da-d83d-4a2c-a727-dc2e1b766e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:45.712248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc905da-d83d-4a2c-a727-dc2e1b766e6c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:45.746234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1092/2203] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:45 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:45.751986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3b134-5fe8-4776-8f71-5d89b023b68a admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:45.761880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3b134-5fe8-4776-8f71-5d89b023b68a admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:45.818035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ad46c-bde6-4895-8083-cc7d61a71e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:45.818691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ad46c-bde6-4895-8083-cc7d61a71e91 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.819004 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ad46c-bde6-4895-8083-cc7d61a71e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.819283 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ad46c-bde6-4895-8083-cc7d61a71e91 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.819959 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ad46c-bde6-4895-8083-cc7d61a71e91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:45.821867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ad46c-bde6-4895-8083-cc7d61a71e91 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:45.840960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1112/2204] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:45 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:45.845017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc905da-d83d-4a2c-a727-dc2e1b766e6c admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:45.855417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc905da-d83d-4a2c-a727-dc2e1b766e6c admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:45.860648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243dd89-ca70-41a0-9644-6e5ce9696340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:45.861296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243dd89-ca70-41a0-9644-6e5ce9696340 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.861585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243dd89-ca70-41a0-9644-6e5ce9696340 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.861862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243dd89-ca70-41a0-9644-6e5ce9696340 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.862590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243dd89-ca70-41a0-9644-6e5ce9696340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:45.864227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243dd89-ca70-41a0-9644-6e5ce9696340 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:45.875149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1093/2205] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:45.878805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ad46c-bde6-4895-8083-cc7d61a71e91 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:45.888573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ad46c-bde6-4895-8083-cc7d61a71e91 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:45.893203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c3d14-0231-4ea6-8ccc-08821326479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:45.893848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3d14-0231-4ea6-8ccc-08821326479e admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.894132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3d14-0231-4ea6-8ccc-08821326479e admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.894410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3d14-0231-4ea6-8ccc-08821326479e admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.895164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c3d14-0231-4ea6-8ccc-08821326479e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:45.896749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c3d14-0231-4ea6-8ccc-08821326479e admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:45.906565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1113/2206] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:45.910510 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243dd89-ca70-41a0-9644-6e5ce9696340 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:45.921192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243dd89-ca70-41a0-9644-6e5ce9696340 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:45.925777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98de7a1-1302-4353-9127-663206e1e19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:45.926644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98de7a1-1302-4353-9127-663206e1e19c admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.927049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98de7a1-1302-4353-9127-663206e1e19c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.927327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98de7a1-1302-4353-9127-663206e1e19c admin admin] PATH_INFO: `/v3/projects/e85459aed5454dafb2461078974aaccf/users/39d506fe431f4f9bbb3a93048f7ce94d/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.942124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98de7a1-1302-4353-9127-663206e1e19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85459aed5454dafb2461078974aaccf, user_id=39d506fe431f4f9bbb3a93048f7ce94d, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:45.944815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98de7a1-1302-4353-9127-663206e1e19c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:45.947827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:45.960792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1094/2207] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:45 2022] PUT /identity/v3/projects/e85459aed5454dafb2461078974aaccf/users/39d506fe431f4f9bbb3a93048f7ce94d/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:45.971366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36ec60-a3ed-44ab-89c2-f16ce853747c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:45.971826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36ec60-a3ed-44ab-89c2-f16ce853747c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:45.971919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36ec60-a3ed-44ab-89c2-f16ce853747c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:45.999081 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-3e36ec60-a3ed-44ab-89c2-f16ce853747c None None] MFA Rules not processed for user `39d506fe431f4f9bbb3a93048f7ce94d`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:46.000296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3e36ec60-a3ed-44ab-89c2-f16ce853747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:46.047101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:46.065519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1114/2208] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:46.077289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98de7a1-1302-4353-9127-663206e1e19c admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:46.079014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c98de7a1-1302-4353-9127-663206e1e19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:46.138277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98de7a1-1302-4353-9127-663206e1e19c admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:46.141949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adea01-eb7e-4a7c-b484-20a414bc0d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:46.142620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adea01-eb7e-4a7c-b484-20a414bc0d63 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:46.142940 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adea01-eb7e-4a7c-b484-20a414bc0d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:46.143219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adea01-eb7e-4a7c-b484-20a414bc0d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:46.153052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adea01-eb7e-4a7c-b484-20a414bc0d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:46.154558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adea01-eb7e-4a7c-b484-20a414bc0d63 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:46.165696 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1095/2209] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:56.062408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:56.161179 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=28 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:56.664465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36ec60-a3ed-44ab-89c2-f16ce853747c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:56.731874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36ec60-a3ed-44ab-89c2-f16ce853747c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:56.737090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:56.737961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:56.738245 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:56.738539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] PATH_INFO: `/v3/users/39d506fe431f4f9bbb3a93048f7ce94d` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:56.739891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d506fe431f4f9bbb3a93048f7ce94d)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:56.741819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:56.792995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d506fe431f4f9bbb3a93048f7ce94d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:56.801252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d506fe431f4f9bbb3a93048f7ce94d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:56.809127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d506fe431f4f9bbb3a93048f7ce94d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:56.809316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d506fe431f4f9bbb3a93048f7ce94d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:56.821801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d506fe431f4f9bbb3a93048f7ce94d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:56.828157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1115/2210] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:51:56 2022] DELETE /identity/v3/users/39d506fe431f4f9bbb3a93048f7ce94d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:56.832679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160d124-a5e8-4952-91c9-409ebd1f0c97 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:56.833043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160d124-a5e8-4952-91c9-409ebd1f0c97 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:56.833330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160d124-a5e8-4952-91c9-409ebd1f0c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:56.851424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-1160d124-a5e8-4952-91c9-409ebd1f0c97 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:56.852637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1160d124-a5e8-4952-91c9-409ebd1f0c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:56.934500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1096/2211] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:56.947866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:56.949295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:57.010718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be4f61-0383-4033-91cc-00bd47377af7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:57.015902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf359a-cba8-4c6c-9e0c-b9178d7f87a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:57.016681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf359a-cba8-4c6c-9e0c-b9178d7f87a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:57.017158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf359a-cba8-4c6c-9e0c-b9178d7f87a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:57.017502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf359a-cba8-4c6c-9e0c-b9178d7f87a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:57.028294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf359a-cba8-4c6c-9e0c-b9178d7f87a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:57.029734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf359a-cba8-4c6c-9e0c-b9178d7f87a0 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:57.042915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1116/2212] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:51:57.742815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160d124-a5e8-4952-91c9-409ebd1f0c97 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:57.744276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1160d124-a5e8-4952-91c9-409ebd1f0c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:57.756924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160d124-a5e8-4952-91c9-409ebd1f0c97 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:57.766100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:57.767095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:57.767378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:57.767671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] PATH_INFO: `/v3/projects/e85459aed5454dafb2461078974aaccf` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:57.768879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e85459aed5454dafb2461078974aaccf)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:57.771982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:57.821213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e85459aed5454dafb2461078974aaccf', 'request_id': 'req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:57.828070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e85459aed5454dafb2461078974aaccf', 'request_id': 'req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:57.835759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e85459aed5454dafb2461078974aaccf', 'request_id': 'req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:57.846751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] The token cache is being invalidate because project e85459aed5454dafb2461078974aaccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:51:57.847072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:57.847698 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-bf0e462b-cb3c-4ab0-90a1-493ab3cd44fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:51:57.849225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1097/2213] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:51:57 2022] DELETE /identity/v3/projects/e85459aed5454dafb2461078974aaccf => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:57.854118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ad63a-641d-40dc-9786-7d37e6b9bd79 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:57.854582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ad63a-641d-40dc-9786-7d37e6b9bd79 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:57.854680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ad63a-641d-40dc-9786-7d37e6b9bd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:57.872606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4d2ad63a-641d-40dc-9786-7d37e6b9bd79 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:57.873803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4d2ad63a-641d-40dc-9786-7d37e6b9bd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:57.891202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:57.940265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1117/2214] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:57.948968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2935a-c9bc-4682-bb7a-ce7767c96f50 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:57.949580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2935a-c9bc-4682-bb7a-ce7767c96f50 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:57.949860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2935a-c9bc-4682-bb7a-ce7767c96f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:57.971482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-3de2935a-c9bc-4682-bb7a-ce7767c96f50 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:57.973006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3de2935a-c9bc-4682-bb7a-ce7767c96f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:57.988290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1098/2215] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:57.989042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ad63a-641d-40dc-9786-7d37e6b9bd79 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.002083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ad63a-641d-40dc-9786-7d37e6b9bd79 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.007151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546a1e9-a36d-462c-a013-07281fe3dff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.008262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a1e9-a36d-462c-a013-07281fe3dff7 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.008262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a1e9-a36d-462c-a013-07281fe3dff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.008462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a1e9-a36d-462c-a013-07281fe3dff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.009185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546a1e9-a36d-462c-a013-07281fe3dff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:51:58.009484 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546a1e9-a36d-462c-a013-07281fe3dff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.012027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546a1e9-a36d-462c-a013-07281fe3dff7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.031518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1118/2216] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:51:58.032829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2935a-c9bc-4682-bb7a-ce7767c96f50 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.055062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2935a-c9bc-4682-bb7a-ce7767c96f50 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.068878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad642e-a9a4-4367-bad3-0d0b189dc751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.069569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad642e-a9a4-4367-bad3-0d0b189dc751 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.069852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad642e-a9a4-4367-bad3-0d0b189dc751 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.070131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad642e-a9a4-4367-bad3-0d0b189dc751 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.071100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad642e-a9a4-4367-bad3-0d0b189dc751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.078996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad642e-a9a4-4367-bad3-0d0b189dc751 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.148084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546a1e9-a36d-462c-a013-07281fe3dff7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.148226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1099/2217] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:58 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:58.161338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546a1e9-a36d-462c-a013-07281fe3dff7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.250765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35702fce-b49a-4a9e-aaa1-eb337f2426e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.251470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35702fce-b49a-4a9e-aaa1-eb337f2426e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.252010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35702fce-b49a-4a9e-aaa1-eb337f2426e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.252301 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35702fce-b49a-4a9e-aaa1-eb337f2426e6 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.253171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35702fce-b49a-4a9e-aaa1-eb337f2426e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.255784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35702fce-b49a-4a9e-aaa1-eb337f2426e6 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.285394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1119/2218] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:58 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:51:58.290195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad642e-a9a4-4367-bad3-0d0b189dc751 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.303200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad642e-a9a4-4367-bad3-0d0b189dc751 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.313159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e470633-5b0b-4c66-b393-dee960f8776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.315140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e470633-5b0b-4c66-b393-dee960f8776e admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.315140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e470633-5b0b-4c66-b393-dee960f8776e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.315319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e470633-5b0b-4c66-b393-dee960f8776e admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.315995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e470633-5b0b-4c66-b393-dee960f8776e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.317664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e470633-5b0b-4c66-b393-dee960f8776e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.333724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1100/2219] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:58.335005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35702fce-b49a-4a9e-aaa1-eb337f2426e6 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.351287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35702fce-b49a-4a9e-aaa1-eb337f2426e6 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.363406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30483823-e566-4fb6-9737-cf46e6070fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.364103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30483823-e566-4fb6-9737-cf46e6070fef admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.364387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30483823-e566-4fb6-9737-cf46e6070fef admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.364689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30483823-e566-4fb6-9737-cf46e6070fef admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.365458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30483823-e566-4fb6-9737-cf46e6070fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.367395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30483823-e566-4fb6-9737-cf46e6070fef admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.379362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1120/2220] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:58.383334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e470633-5b0b-4c66-b393-dee960f8776e admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.395385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e470633-5b0b-4c66-b393-dee960f8776e admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.401469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423de6-315c-4c78-bf69-2ad6233610bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.402549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423de6-315c-4c78-bf69-2ad6233610bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.402653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423de6-315c-4c78-bf69-2ad6233610bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.402948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423de6-315c-4c78-bf69-2ad6233610bd admin admin] PATH_INFO: `/v3/projects/572a2e2ba29643a1aea56f2a7b7881c7/users/481dc4e5e20c4cb7bca3a07fcb45b35b/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.420372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72423de6-315c-4c78-bf69-2ad6233610bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=572a2e2ba29643a1aea56f2a7b7881c7, user_id=481dc4e5e20c4cb7bca3a07fcb45b35b, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.423661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72423de6-315c-4c78-bf69-2ad6233610bd admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.443498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1101/2221] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:58 2022] PUT /identity/v3/projects/572a2e2ba29643a1aea56f2a7b7881c7/users/481dc4e5e20c4cb7bca3a07fcb45b35b/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:51:58.454636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088121e-4eb4-4b27-af64-a8db83cd87d6 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.455361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088121e-4eb4-4b27-af64-a8db83cd87d6 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.455682 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088121e-4eb4-4b27-af64-a8db83cd87d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.484490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b088121e-4eb4-4b27-af64-a8db83cd87d6 None None] MFA Rules not processed for user `481dc4e5e20c4cb7bca3a07fcb45b35b`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:58.485731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b088121e-4eb4-4b27-af64-a8db83cd87d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:58.549480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1121/2222] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:58.554067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423de6-315c-4c78-bf69-2ad6233610bd admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.574978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:58.617024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423de6-315c-4c78-bf69-2ad6233610bd admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.621603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c440c-755e-48f5-9fbf-ba736de0cdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.622250 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c440c-755e-48f5-9fbf-ba736de0cdab admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.622530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c440c-755e-48f5-9fbf-ba736de0cdab admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.622843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c440c-755e-48f5-9fbf-ba736de0cdab admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.623521 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c440c-755e-48f5-9fbf-ba736de0cdab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.626023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c440c-755e-48f5-9fbf-ba736de0cdab admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.649444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1102/2223] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:58 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:58.653505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088121e-4eb4-4b27-af64-a8db83cd87d6 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.664568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088121e-4eb4-4b27-af64-a8db83cd87d6 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.676995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:58.722135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd5fb7-6b01-4952-b293-330d3b12e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.722822 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd5fb7-6b01-4952-b293-330d3b12e508 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.723109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd5fb7-6b01-4952-b293-330d3b12e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.723387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd5fb7-6b01-4952-b293-330d3b12e508 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.724080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd5fb7-6b01-4952-b293-330d3b12e508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.725996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd5fb7-6b01-4952-b293-330d3b12e508 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.746921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1122/2224] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:58 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:58.751345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c440c-755e-48f5-9fbf-ba736de0cdab admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.762087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c440c-755e-48f5-9fbf-ba736de0cdab admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.766690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8185e-9c08-4fe4-810f-8f6b5799eb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.767422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8185e-9c08-4fe4-810f-8f6b5799eb75 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.767702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8185e-9c08-4fe4-810f-8f6b5799eb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.767978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8185e-9c08-4fe4-810f-8f6b5799eb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.768693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8185e-9c08-4fe4-810f-8f6b5799eb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.770283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8185e-9c08-4fe4-810f-8f6b5799eb75 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.781830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1103/2225] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:58.790717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd5fb7-6b01-4952-b293-330d3b12e508 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.796882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd5fb7-6b01-4952-b293-330d3b12e508 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.801545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8baf3-4a86-494c-b943-b859c0dbb77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.802198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8baf3-4a86-494c-b943-b859c0dbb77c admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.802482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8baf3-4a86-494c-b943-b859c0dbb77c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.802785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8baf3-4a86-494c-b943-b859c0dbb77c admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.803514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8baf3-4a86-494c-b943-b859c0dbb77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.805156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8baf3-4a86-494c-b943-b859c0dbb77c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.815026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1123/2226] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:58.819340 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8185e-9c08-4fe4-810f-8f6b5799eb75 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:58.837537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8185e-9c08-4fe4-810f-8f6b5799eb75 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:58.842256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1436638-aceb-42e3-94bc-27ea6d727194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:58.843702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1436638-aceb-42e3-94bc-27ea6d727194 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.843985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1436638-aceb-42e3-94bc-27ea6d727194 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.844260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1436638-aceb-42e3-94bc-27ea6d727194 admin admin] PATH_INFO: `/v3/projects/6be785ce0f3d4f3784348f677ed0526a/users/8b285a19b7204220bb19b8f54fb5c2a0/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.859380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1436638-aceb-42e3-94bc-27ea6d727194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be785ce0f3d4f3784348f677ed0526a, user_id=8b285a19b7204220bb19b8f54fb5c2a0, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:58.862008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1436638-aceb-42e3-94bc-27ea6d727194 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:58.874645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1104/2227] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:58 2022] PUT /identity/v3/projects/6be785ce0f3d4f3784348f677ed0526a/users/8b285a19b7204220bb19b8f54fb5c2a0/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:58.884879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba4461-f558-4410-aefd-5ff8641f0ec2 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:58.885328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba4461-f558-4410-aefd-5ff8641f0ec2 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:58.885443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba4461-f558-4410-aefd-5ff8641f0ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:58.911668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-64ba4461-f558-4410-aefd-5ff8641f0ec2 None None] MFA Rules not processed for user `8b285a19b7204220bb19b8f54fb5c2a0`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:58.913112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-64ba4461-f558-4410-aefd-5ff8641f0ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:58.975760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1124/2228] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:58.980262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1436638-aceb-42e3-94bc-27ea6d727194 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:59.040819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1436638-aceb-42e3-94bc-27ea6d727194 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:59.045490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474b40-da58-49ed-be2a-a85d1f42c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:59.046197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474b40-da58-49ed-be2a-a85d1f42c147 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:59.047014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474b40-da58-49ed-be2a-a85d1f42c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:59.047281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474b40-da58-49ed-be2a-a85d1f42c147 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:59.047957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d474b40-da58-49ed-be2a-a85d1f42c147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:59.049854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d474b40-da58-49ed-be2a-a85d1f42c147 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:59.074619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1105/2229] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:51:58 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:59.078673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba4461-f558-4410-aefd-5ff8641f0ec2 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:59.089506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba4461-f558-4410-aefd-5ff8641f0ec2 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:59.146564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4466a-bee4-41dc-9c70-49afe19aa9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:59.147275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4466a-bee4-41dc-9c70-49afe19aa9ff admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:59.147560 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4466a-bee4-41dc-9c70-49afe19aa9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:59.147865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4466a-bee4-41dc-9c70-49afe19aa9ff admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:59.148567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4466a-bee4-41dc-9c70-49afe19aa9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:59.150494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4466a-bee4-41dc-9c70-49afe19aa9ff admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:59.172285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1125/2230] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:51:59 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:51:59.176302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474b40-da58-49ed-be2a-a85d1f42c147 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:59.186477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474b40-da58-49ed-be2a-a85d1f42c147 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:59.191080 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed370e-a805-46bc-b199-592cb763757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:59.191753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed370e-a805-46bc-b199-592cb763757f admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:59.192042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed370e-a805-46bc-b199-592cb763757f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:59.192321 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed370e-a805-46bc-b199-592cb763757f admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:59.193084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed370e-a805-46bc-b199-592cb763757f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:59.194678 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed370e-a805-46bc-b199-592cb763757f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:59.206069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1106/2231] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:59.210177 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4466a-bee4-41dc-9c70-49afe19aa9ff admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:59.220666 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4466a-bee4-41dc-9c70-49afe19aa9ff admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:59.225288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55b650-31ad-4630-be0a-84ed9cf85a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:59.226161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b650-31ad-4630-be0a-84ed9cf85a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:59.226443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b650-31ad-4630-be0a-84ed9cf85a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:59.226720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b650-31ad-4630-be0a-84ed9cf85a91 admin admin] PATH_INFO: `/v3/projects/e4fdfd66e8784f84a86f57a8b6a642f4/users/b4d97d374ca545ad90a6becbf3de20b1/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:59.239311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:59.242947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55b650-31ad-4630-be0a-84ed9cf85a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fdfd66e8784f84a86f57a8b6a642f4, user_id=b4d97d374ca545ad90a6becbf3de20b1, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:59.246302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55b650-31ad-4630-be0a-84ed9cf85a91 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:59.260505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1126/2232] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:59 2022] PUT /identity/v3/projects/e4fdfd66e8784f84a86f57a8b6a642f4/users/b4d97d374ca545ad90a6becbf3de20b1/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:59.264429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed370e-a805-46bc-b199-592cb763757f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:59.275924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed370e-a805-46bc-b199-592cb763757f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:59.319809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:51:59.335131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2b3be-1833-4259-8669-d9096db2e761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:59.335738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2b3be-1833-4259-8669-d9096db2e761 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:59.336022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2b3be-1833-4259-8669-d9096db2e761 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:59.336297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2b3be-1833-4259-8669-d9096db2e761 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:59.337014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2b3be-1833-4259-8669-d9096db2e761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:59.338601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2b3be-1833-4259-8669-d9096db2e761 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:59.348136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1107/2233] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:51:59.352188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55b650-31ad-4630-be0a-84ed9cf85a91 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:59.362651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55b650-31ad-4630-be0a-84ed9cf85a91 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:59.367478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d39007-663f-414a-aabf-73901a3ba277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:59.368353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d39007-663f-414a-aabf-73901a3ba277 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:59.368632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d39007-663f-414a-aabf-73901a3ba277 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:59.369210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d39007-663f-414a-aabf-73901a3ba277 admin admin] PATH_INFO: `/v3/projects/e4fdfd66e8784f84a86f57a8b6a642f4/users/b4d97d374ca545ad90a6becbf3de20b1/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:59.370911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d39007-663f-414a-aabf-73901a3ba277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fdfd66e8784f84a86f57a8b6a642f4, user_id=b4d97d374ca545ad90a6becbf3de20b1, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:59.373555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d39007-663f-414a-aabf-73901a3ba277 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:59.384520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1127/2234] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:51:59 2022] PUT /identity/v3/projects/e4fdfd66e8784f84a86f57a8b6a642f4/users/b4d97d374ca545ad90a6becbf3de20b1/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:51:59.395702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739183c-26bf-4444-9f8a-77c4f9467966 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:59.396072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739183c-26bf-4444-9f8a-77c4f9467966 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:59.396356 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739183c-26bf-4444-9f8a-77c4f9467966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:59.428630 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9739183c-26bf-4444-9f8a-77c4f9467966 None None] MFA Rules not processed for user `b4d97d374ca545ad90a6becbf3de20b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:51:59.430108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9739183c-26bf-4444-9f8a-77c4f9467966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:59.515375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1108/2235] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:51:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:51:59.526604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d39007-663f-414a-aabf-73901a3ba277 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:51:59.528427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-08d39007-663f-414a-aabf-73901a3ba277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:51:59.594471 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d39007-663f-414a-aabf-73901a3ba277 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:51:59.598139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485527b-773d-435b-9f08-8d6b0dc62ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:51:59.598808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485527b-773d-435b-9f08-8d6b0dc62ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:51:59.599091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485527b-773d-435b-9f08-8d6b0dc62ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:51:59.599363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485527b-773d-435b-9f08-8d6b0dc62ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:51:59.656055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485527b-773d-435b-9f08-8d6b0dc62ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:51:59.657494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485527b-773d-435b-9f08-8d6b0dc62ec2 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:51:59.666532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1128/2236] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:51:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:52:07.448781 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739183c-26bf-4444-9f8a-77c4f9467966 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:07.450255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9739183c-26bf-4444-9f8a-77c4f9467966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:07.461997 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739183c-26bf-4444-9f8a-77c4f9467966 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:07.465579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f2d9f-70ef-40e2-8503-5f6bb03f09c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:07.466218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f2d9f-70ef-40e2-8503-5f6bb03f09c5 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:07.466504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f2d9f-70ef-40e2-8503-5f6bb03f09c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:07.466813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f2d9f-70ef-40e2-8503-5f6bb03f09c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:07.522562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f2d9f-70ef-40e2-8503-5f6bb03f09c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:07.524017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f2d9f-70ef-40e2-8503-5f6bb03f09c5 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:07.532932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1109/2237] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:52:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:52:08.116942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485527b-773d-435b-9f08-8d6b0dc62ec2 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:08.126338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485527b-773d-435b-9f08-8d6b0dc62ec2 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:08.130343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dd8c0-65ed-4d97-88ed-ae1566a64181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:08.131053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd8c0-65ed-4d97-88ed-ae1566a64181 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:08.131339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd8c0-65ed-4d97-88ed-ae1566a64181 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:08.131617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd8c0-65ed-4d97-88ed-ae1566a64181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:08.141806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dd8c0-65ed-4d97-88ed-ae1566a64181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:08.143384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dd8c0-65ed-4d97-88ed-ae1566a64181 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:08.154349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1129/2238] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:52:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:52:13.262137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f2d9f-70ef-40e2-8503-5f6bb03f09c5 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:13.335341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f2d9f-70ef-40e2-8503-5f6bb03f09c5 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:13.339873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:13.340752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:13.341035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:13.341314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] PATH_INFO: `/v3/users/481dc4e5e20c4cb7bca3a07fcb45b35b` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:13.342494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=481dc4e5e20c4cb7bca3a07fcb45b35b)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:13.344903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:13.350154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:13.395762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481dc4e5e20c4cb7bca3a07fcb45b35b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.402840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481dc4e5e20c4cb7bca3a07fcb45b35b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.410245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481dc4e5e20c4cb7bca3a07fcb45b35b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.410910 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481dc4e5e20c4cb7bca3a07fcb45b35b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.411518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481dc4e5e20c4cb7bca3a07fcb45b35b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.430065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1110/2239] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:52:13 2022] DELETE /identity/v3/users/481dc4e5e20c4cb7bca3a07fcb45b35b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:13.434482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc43881-ed90-42be-99a1-dd556a6c953f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:13.434993 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc43881-ed90-42be-99a1-dd556a6c953f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:13.435079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc43881-ed90-42be-99a1-dd556a6c953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:13.457796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-bfc43881-ed90-42be-99a1-dd556a6c953f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:13.459534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bfc43881-ed90-42be-99a1-dd556a6c953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:13.525589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1130/2240] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:13.532516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:13.534057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:13.595897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f7c11-b930-43ad-b3e6-1bac0360e0c0 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:13.601602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b14e-6050-459c-896e-9243bf5d9289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:13.602276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b14e-6050-459c-896e-9243bf5d9289 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:13.602548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b14e-6050-459c-896e-9243bf5d9289 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:13.602853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b14e-6050-459c-896e-9243bf5d9289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:13.613267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1b14e-6050-459c-896e-9243bf5d9289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:13.614691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1b14e-6050-459c-896e-9243bf5d9289 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:13.624291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1111/2241] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:52:13.886533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc43881-ed90-42be-99a1-dd556a6c953f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:13.888017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bfc43881-ed90-42be-99a1-dd556a6c953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:13.899964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc43881-ed90-42be-99a1-dd556a6c953f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:13.905489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:13.906353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:13.907176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:13.907445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] PATH_INFO: `/v3/projects/572a2e2ba29643a1aea56f2a7b7881c7` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:13.908638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=572a2e2ba29643a1aea56f2a7b7881c7)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:13.911455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:13.929933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:13.961357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572a2e2ba29643a1aea56f2a7b7881c7', 'request_id': 'req-7d81e61b-b56c-4a02-8f14-5303327e6cd4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.967459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572a2e2ba29643a1aea56f2a7b7881c7', 'request_id': 'req-7d81e61b-b56c-4a02-8f14-5303327e6cd4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.975156 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572a2e2ba29643a1aea56f2a7b7881c7', 'request_id': 'req-7d81e61b-b56c-4a02-8f14-5303327e6cd4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.985851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] The token cache is being invalidate because project 572a2e2ba29643a1aea56f2a7b7881c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:52:13.986154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.986769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:13.988305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1131/2242] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:52:13 2022] DELETE /identity/v3/projects/572a2e2ba29643a1aea56f2a7b7881c7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:52:13.992486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b14e-6050-459c-896e-9243bf5d9289 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:14.002165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b14e-6050-459c-896e-9243bf5d9289 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:14.059303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:14.060099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:14.060380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:14.060655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] PATH_INFO: `/v3/users/8b285a19b7204220bb19b8f54fb5c2a0` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:14.061848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b285a19b7204220bb19b8f54fb5c2a0)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:14.063809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:14.090960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:14.111471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b285a19b7204220bb19b8f54fb5c2a0'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.118390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b285a19b7204220bb19b8f54fb5c2a0'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.125246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b285a19b7204220bb19b8f54fb5c2a0'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.125854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b285a19b7204220bb19b8f54fb5c2a0'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.126456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b285a19b7204220bb19b8f54fb5c2a0'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.143810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1112/2243] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:52:13 2022] DELETE /identity/v3/users/8b285a19b7204220bb19b8f54fb5c2a0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:14.412652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:14.414188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:14.429230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81e61b-b56c-4a02-8f14-5303327e6cd4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:14.495232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:14.495232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:14.495232 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:14.496026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] PATH_INFO: `/v3/projects/6be785ce0f3d4f3784348f677ed0526a` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:14.496909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be785ce0f3d4f3784348f677ed0526a)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:14.498909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:14.555819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be785ce0f3d4f3784348f677ed0526a', 'request_id': 'req-c981feec-3514-4323-a4aa-c6e69b9227e4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.562167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be785ce0f3d4f3784348f677ed0526a', 'request_id': 'req-c981feec-3514-4323-a4aa-c6e69b9227e4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.570981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be785ce0f3d4f3784348f677ed0526a', 'request_id': 'req-c981feec-3514-4323-a4aa-c6e69b9227e4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.583811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] The token cache is being invalidate because project 6be785ce0f3d4f3784348f677ed0526a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:52:14.584033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.584688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.586272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1132/2244] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:52:14 2022] DELETE /identity/v3/projects/6be785ce0f3d4f3784348f677ed0526a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:14.590174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:14.591697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:14.604633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc03ea-791c-4a24-923e-114574d74445 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:14.665518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:14.666439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:14.666742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:14.667032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] PATH_INFO: `/v3/users/b4d97d374ca545ad90a6becbf3de20b1` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:14.668235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d97d374ca545ad90a6becbf3de20b1)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:14.670166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:14.731184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d97d374ca545ad90a6becbf3de20b1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.738841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d97d374ca545ad90a6becbf3de20b1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.746166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d97d374ca545ad90a6becbf3de20b1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.746883 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d97d374ca545ad90a6becbf3de20b1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.747619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d97d374ca545ad90a6becbf3de20b1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:14.765428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1113/2245] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:52:14 2022] DELETE /identity/v3/users/b4d97d374ca545ad90a6becbf3de20b1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:15.087338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:15.088846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:15.126413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981feec-3514-4323-a4aa-c6e69b9227e4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:15.135847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:15.249667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:15.251058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:15.251338 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:15.251615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] PATH_INFO: `/v3/projects/e4fdfd66e8784f84a86f57a8b6a642f4` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:15.252797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fdfd66e8784f84a86f57a8b6a642f4)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:15.254700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:15.300674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fdfd66e8784f84a86f57a8b6a642f4', 'request_id': 'req-387329fe-22f9-430b-b1d8-584cadacba85'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:15.306607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fdfd66e8784f84a86f57a8b6a642f4', 'request_id': 'req-387329fe-22f9-430b-b1d8-584cadacba85'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:15.314307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fdfd66e8784f84a86f57a8b6a642f4', 'request_id': 'req-387329fe-22f9-430b-b1d8-584cadacba85'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:15.325178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] The token cache is being invalidate because project e4fdfd66e8784f84a86f57a8b6a642f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:52:15.325466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:15.326102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-387329fe-22f9-430b-b1d8-584cadacba85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:15.327656 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1133/2246] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:52:15 2022] DELETE /identity/v3/projects/e4fdfd66e8784f84a86f57a8b6a642f4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:23.125108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:23.127243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:23.165685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:23.217868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efffe7-a3e0-4fad-a213-33cc56c8992b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:23.292905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:23.294178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:23.294327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:23.294714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] PATH_INFO: `/v3/users/27737b6037514a908fe209f41b22e293` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:23.296190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27737b6037514a908fe209f41b22e293)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:23.298710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:23.371605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27737b6037514a908fe209f41b22e293'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:23.378357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27737b6037514a908fe209f41b22e293'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:23.388313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27737b6037514a908fe209f41b22e293'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:23.389093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27737b6037514a908fe209f41b22e293'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:23.389744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27737b6037514a908fe209f41b22e293'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:23.413445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1114/2247] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:52:23 2022] DELETE /identity/v3/users/27737b6037514a908fe209f41b22e293 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:23.416435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28159a2-63cd-4d7d-bcca-0cd551050f27 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:23.416906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28159a2-63cd-4d7d-bcca-0cd551050f27 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:23.416985 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28159a2-63cd-4d7d-bcca-0cd551050f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:23.448810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-c28159a2-63cd-4d7d-bcca-0cd551050f27 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:23.452440 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c28159a2-63cd-4d7d-bcca-0cd551050f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:23.513086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:23.531001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1134/2248] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:23.542017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:23.543579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:23.563803 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f255519-6077-4abf-91ca-a0c52e6686e4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:23.564094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f255519-6077-4abf-91ca-a0c52e6686e4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:23.564392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f255519-6077-4abf-91ca-a0c52e6686e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:23.584430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6f255519-6077-4abf-91ca-a0c52e6686e4 None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:23.585916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6f255519-6077-4abf-91ca-a0c52e6686e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:23.621272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:23.653306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1920e4-907d-4fbe-981c-4a0a3f446068 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:23.654893 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1135/2249] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:52:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:23.660703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b4ff1-2ec6-4cf0-82d3-52f24f8042e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:23.661443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b4ff1-2ec6-4cf0-82d3-52f24f8042e0 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:23.661813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b4ff1-2ec6-4cf0-82d3-52f24f8042e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:23.662430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b4ff1-2ec6-4cf0-82d3-52f24f8042e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:23.682689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b4ff1-2ec6-4cf0-82d3-52f24f8042e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:23.684498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b4ff1-2ec6-4cf0-82d3-52f24f8042e0 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:23.700703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1115/2250] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:52:23.707249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f255519-6077-4abf-91ca-a0c52e6686e4 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:23.709070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6f255519-6077-4abf-91ca-a0c52e6686e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:23.790284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f255519-6077-4abf-91ca-a0c52e6686e4 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:23.795724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07c73b-a53c-41b3-b5e2-a23340701fb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:23.796420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07c73b-a53c-41b3-b5e2-a23340701fb0 service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:23.796770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07c73b-a53c-41b3-b5e2-a23340701fb0 service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:23.797113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07c73b-a53c-41b3-b5e2-a23340701fb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:23.808927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07c73b-a53c-41b3-b5e2-a23340701fb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:23.810540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07c73b-a53c-41b3-b5e2-a23340701fb0 service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:23.820087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1136/2251] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:52:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:52:23.984301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b4ff1-2ec6-4cf0-82d3-52f24f8042e0 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:23.985766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-478b4ff1-2ec6-4cf0-82d3-52f24f8042e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:24.001438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b4ff1-2ec6-4cf0-82d3-52f24f8042e0 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.006396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.007298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.007582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.007857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] PATH_INFO: `/v3/projects/047da195e943441fabfc3ab9d7eb420f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.009060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] RBAC: Authorizing `identity:delete_project(project_id=047da195e943441fabfc3ab9d7eb420f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.010996 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:24.067600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '047da195e943441fabfc3ab9d7eb420f', 'request_id': 'req-021291c3-db91-40e1-8144-eb01396da98c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:24.076905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '047da195e943441fabfc3ab9d7eb420f', 'request_id': 'req-021291c3-db91-40e1-8144-eb01396da98c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:24.085633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '047da195e943441fabfc3ab9d7eb420f', 'request_id': 'req-021291c3-db91-40e1-8144-eb01396da98c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:24.100115 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] The token cache is being invalidate because project 047da195e943441fabfc3ab9d7eb420f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:52:24.100409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:24.101097 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-021291c3-db91-40e1-8144-eb01396da98c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:24.102802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1116/2252] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:52:23 2022] DELETE /identity/v3/projects/047da195e943441fabfc3ab9d7eb420f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:52:24.107638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2f8ed-3ba8-46da-a0a9-5c000615fdbf None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.107924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2f8ed-3ba8-46da-a0a9-5c000615fdbf None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.108198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2f8ed-3ba8-46da-a0a9-5c000615fdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.150601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-52a2f8ed-3ba8-46da-a0a9-5c000615fdbf None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:24.154053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-52a2f8ed-3ba8-46da-a0a9-5c000615fdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:24.231455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1137/2253] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:24.238901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad524c9-80e9-47cd-a13a-aec0741d363b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.239388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad524c9-80e9-47cd-a13a-aec0741d363b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.239483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad524c9-80e9-47cd-a13a-aec0741d363b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.259518 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-cad524c9-80e9-47cd-a13a-aec0741d363b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:24.260711 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cad524c9-80e9-47cd-a13a-aec0741d363b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:24.269730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1117/2254] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:24.273892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2f8ed-3ba8-46da-a0a9-5c000615fdbf None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.284406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2f8ed-3ba8-46da-a0a9-5c000615fdbf None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.289583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f4c47-db93-4127-bbc7-55735d44cb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.290360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f4c47-db93-4127-bbc7-55735d44cb82 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.290655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f4c47-db93-4127-bbc7-55735d44cb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.291396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f4c47-db93-4127-bbc7-55735d44cb82 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.291676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f4c47-db93-4127-bbc7-55735d44cb82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:52:24.292078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f4c47-db93-4127-bbc7-55735d44cb82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.293680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f4c47-db93-4127-bbc7-55735d44cb82 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:24.306761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1138/2255] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:52:24.310401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad524c9-80e9-47cd-a13a-aec0741d363b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.320643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad524c9-80e9-47cd-a13a-aec0741d363b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.325414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fad0ff-b912-437e-88b4-a95323cff6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.326088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fad0ff-b912-437e-88b4-a95323cff6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.326362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fad0ff-b912-437e-88b4-a95323cff6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.326638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fad0ff-b912-437e-88b4-a95323cff6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.327370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fad0ff-b912-437e-88b4-a95323cff6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.329856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fad0ff-b912-437e-88b4-a95323cff6d4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:24.356256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1118/2256] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:52:24 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:52:24.360309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f4c47-db93-4127-bbc7-55735d44cb82 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.365443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:24.371632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f4c47-db93-4127-bbc7-55735d44cb82 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.385837 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a16dd-80dd-4e49-8a7b-ab6b4d67c8ef None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.386137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a16dd-80dd-4e49-8a7b-ab6b4d67c8ef None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.386427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a16dd-80dd-4e49-8a7b-ab6b4d67c8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.406961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-057a16dd-80dd-4e49-8a7b-ab6b4d67c8ef None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:24.408178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-057a16dd-80dd-4e49-8a7b-ab6b4d67c8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:24.452747 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e792e39-4aef-48cc-90bd-ac3947cac735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.453327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e792e39-4aef-48cc-90bd-ac3947cac735 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.453670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e792e39-4aef-48cc-90bd-ac3947cac735 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.454825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e792e39-4aef-48cc-90bd-ac3947cac735 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.455306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e792e39-4aef-48cc-90bd-ac3947cac735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.457243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e792e39-4aef-48cc-90bd-ac3947cac735 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:24.470707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1119/2257] 10.222.0.62 () {58 vars in 967 bytes} [Wed Jun 15 15:52:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:24.480449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1139/2258] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:52:24 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:52:24.484367 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a16dd-80dd-4e49-8a7b-ab6b4d67c8ef None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.487711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e792e39-4aef-48cc-90bd-ac3947cac735 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.490203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3e792e39-4aef-48cc-90bd-ac3947cac735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:24.496961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a16dd-80dd-4e49-8a7b-ab6b4d67c8ef None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.503331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92d509-9b63-4fc2-a12c-6cf76c67c594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.504121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92d509-9b63-4fc2-a12c-6cf76c67c594 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.504700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92d509-9b63-4fc2-a12c-6cf76c67c594 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.505113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92d509-9b63-4fc2-a12c-6cf76c67c594 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.505939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92d509-9b63-4fc2-a12c-6cf76c67c594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.507675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92d509-9b63-4fc2-a12c-6cf76c67c594 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:24.519734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1120/2259] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:52:24.523958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92d509-9b63-4fc2-a12c-6cf76c67c594 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.534864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92d509-9b63-4fc2-a12c-6cf76c67c594 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.540900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e0dd9-69e4-4c40-853d-921ed5369bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.541928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e0dd9-69e4-4c40-853d-921ed5369bfb admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.542419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e0dd9-69e4-4c40-853d-921ed5369bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.542816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e0dd9-69e4-4c40-853d-921ed5369bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.543650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e0dd9-69e4-4c40-853d-921ed5369bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.545569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e0dd9-69e4-4c40-853d-921ed5369bfb admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:24.558294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1121/2260] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:52:24.562136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e0dd9-69e4-4c40-853d-921ed5369bfb admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.570288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e792e39-4aef-48cc-90bd-ac3947cac735 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.572247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e0dd9-69e4-4c40-853d-921ed5369bfb admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.575458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04362-24b5-4602-8ef1-176b5ae0bb3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.576106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04362-24b5-4602-8ef1-176b5ae0bb3c service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.576392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04362-24b5-4602-8ef1-176b5ae0bb3c service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.576673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04362-24b5-4602-8ef1-176b5ae0bb3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.576796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fa4c3-24c9-484f-9d6f-dff9873b0c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.577655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fa4c3-24c9-484f-9d6f-dff9873b0c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.577933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fa4c3-24c9-484f-9d6f-dff9873b0c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.578204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fa4c3-24c9-484f-9d6f-dff9873b0c84 admin admin] PATH_INFO: `/v3/projects/e59e1b9b7f884f4ea041b273d2aa6dca/users/a579e255cbf345958e7dbe484d599611/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.580976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:24.585966 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca04362-24b5-4602-8ef1-176b5ae0bb3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.587660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca04362-24b5-4602-8ef1-176b5ae0bb3c service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:24.595736 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fa4c3-24c9-484f-9d6f-dff9873b0c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e59e1b9b7f884f4ea041b273d2aa6dca, user_id=a579e255cbf345958e7dbe484d599611, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.596291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1140/2261] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:52:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:52:24.598346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fa4c3-24c9-484f-9d6f-dff9873b0c84 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:24.612344 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1122/2262] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:52:24 2022] PUT /identity/v3/projects/e59e1b9b7f884f4ea041b273d2aa6dca/users/a579e255cbf345958e7dbe484d599611/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:52:24.621922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c839f5-b8aa-4710-bd43-7335a84cd37e None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.622212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c839f5-b8aa-4710-bd43-7335a84cd37e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.622489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c839f5-b8aa-4710-bd43-7335a84cd37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.650357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-86c839f5-b8aa-4710-bd43-7335a84cd37e None None] MFA Rules not processed for user `a579e255cbf345958e7dbe484d599611`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:24.651756 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-86c839f5-b8aa-4710-bd43-7335a84cd37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:24.721478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1141/2263] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:24.734011 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fa4c3-24c9-484f-9d6f-dff9873b0c84 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.734670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b75fa4c3-24c9-484f-9d6f-dff9873b0c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:24.806753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fa4c3-24c9-484f-9d6f-dff9873b0c84 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.810896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c696f-e068-4561-914f-21631a9c322f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.811561 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c696f-e068-4561-914f-21631a9c322f service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.811826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c696f-e068-4561-914f-21631a9c322f service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.811929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c003fa-eb51-4d68-864f-e72afe514b92 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.812099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c696f-e068-4561-914f-21631a9c322f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.812209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c003fa-eb51-4d68-864f-e72afe514b92 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.812565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c003fa-eb51-4d68-864f-e72afe514b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.823558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c696f-e068-4561-914f-21631a9c322f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.825002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c696f-e068-4561-914f-21631a9c322f service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:24.832669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-74c003fa-eb51-4d68-864f-e72afe514b92 None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:24.833879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-74c003fa-eb51-4d68-864f-e72afe514b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:24.834694 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1123/2264] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:52:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:52:24.839926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:24.868713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c696f-e068-4561-914f-21631a9c322f service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.932432 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1142/2265] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:52:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:52:24.944658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c003fa-eb51-4d68-864f-e72afe514b92 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:24.950338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c696f-e068-4561-914f-21631a9c322f service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:24.955034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:24.955898 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:24.956174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:24.956446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] PATH_INFO: `/v3/users/a579e255cbf345958e7dbe484d599611` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:24.957627 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a579e255cbf345958e7dbe484d599611)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:24.959556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:25.011797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a579e255cbf345958e7dbe484d599611'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.019476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a579e255cbf345958e7dbe484d599611'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.019762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c003fa-eb51-4d68-864f-e72afe514b92 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:25.025283 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0fe9d-b0cd-4999-bcee-71e9d2129519 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:25.026009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0fe9d-b0cd-4999-bcee-71e9d2129519 service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:25.026293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0fe9d-b0cd-4999-bcee-71e9d2129519 service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:25.026573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0fe9d-b0cd-4999-bcee-71e9d2129519 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:25.026984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a579e255cbf345958e7dbe484d599611'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.027647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a579e255cbf345958e7dbe484d599611'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.028252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-b2785f64-ccd4-4996-86e0-b7d56a47c726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a579e255cbf345958e7dbe484d599611'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.036269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0fe9d-b0cd-4999-bcee-71e9d2129519 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:25.038125 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0fe9d-b0cd-4999-bcee-71e9d2129519 service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:25.048156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1124/2266] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:52:24 2022] DELETE /identity/v3/users/a579e255cbf345958e7dbe484d599611 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:25.051326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1143/2267] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:52:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 15:52:25.052224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c152a-4748-4213-935a-6f9c73ac7e52 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:25.052654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c152a-4748-4213-935a-6f9c73ac7e52 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:25.053082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c152a-4748-4213-935a-6f9c73ac7e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:25.061072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:25.072701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-6b4c152a-4748-4213-935a-6f9c73ac7e52 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:25.075016 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6b4c152a-4748-4213-935a-6f9c73ac7e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:25.142751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1125/2268] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:52:25.152264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0fe9d-b0cd-4999-bcee-71e9d2129519 service swift] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:25.153971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-56e0fe9d-b0cd-4999-bcee-71e9d2129519 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:25.226007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0fe9d-b0cd-4999-bcee-71e9d2129519 service swift] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:25.278817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f6030-4167-4bd4-b51a-92d8a64292cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:25.279699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f6030-4167-4bd4-b51a-92d8a64292cf service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:25.279990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f6030-4167-4bd4-b51a-92d8a64292cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:25.280317 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f6030-4167-4bd4-b51a-92d8a64292cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:25.293627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f6030-4167-4bd4-b51a-92d8a64292cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:25.295254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f6030-4167-4bd4-b51a-92d8a64292cf service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:25.306703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1144/2269] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:52:25.461822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbb345-35d8-42c9-9acc-c91fb006bc60 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:25.462225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbb345-35d8-42c9-9acc-c91fb006bc60 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:25.462537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbb345-35d8-42c9-9acc-c91fb006bc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:25.482247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-10bbb345-35d8-42c9-9acc-c91fb006bc60 None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:25.483600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-10bbb345-35d8-42c9-9acc-c91fb006bc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:25.549422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1126/2270] 10.222.0.62 () {58 vars in 967 bytes} [Wed Jun 15 15:52:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:52:25.562593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f6030-4167-4bd4-b51a-92d8a64292cf service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:25.564202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-624f6030-4167-4bd4-b51a-92d8a64292cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:25.570834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:25.636107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbb345-35d8-42c9-9acc-c91fb006bc60 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:25.637685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-10bbb345-35d8-42c9-9acc-c91fb006bc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:25.658129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbb345-35d8-42c9-9acc-c91fb006bc60 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:25.666108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:25.667038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:25.667347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:25.667960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] PATH_INFO: `/v3/projects/e59e1b9b7f884f4ea041b273d2aa6dca` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:25.669704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e59e1b9b7f884f4ea041b273d2aa6dca)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:25.669839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f6030-4167-4bd4-b51a-92d8a64292cf service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:25.671435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:25.675764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01fa40-e6a6-4189-ac60-50b9382056c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:25.676553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01fa40-e6a6-4189-ac60-50b9382056c1 service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:25.676941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01fa40-e6a6-4189-ac60-50b9382056c1 service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:25.677322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01fa40-e6a6-4189-ac60-50b9382056c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:25.687980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01fa40-e6a6-4189-ac60-50b9382056c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:25.689636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01fa40-e6a6-4189-ac60-50b9382056c1 service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:25.704273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1145/2271] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:52:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:52:25.726939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e59e1b9b7f884f4ea041b273d2aa6dca', 'request_id': 'req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.734267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e59e1b9b7f884f4ea041b273d2aa6dca', 'request_id': 'req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.742933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e59e1b9b7f884f4ea041b273d2aa6dca', 'request_id': 'req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.764167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] The token cache is being invalidate because project e59e1b9b7f884f4ea041b273d2aa6dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:52:25.764519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.765486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-51827ce6-22f2-4e31-bbda-b8f9c4ed0f7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:25.768153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1127/2272] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:52:25 2022] DELETE /identity/v3/projects/e59e1b9b7f884f4ea041b273d2aa6dca => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:52:27.934464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01fa40-e6a6-4189-ac60-50b9382056c1 service swift] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:27.935935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1a01fa40-e6a6-4189-ac60-50b9382056c1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:27.949240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01fa40-e6a6-4189-ac60-50b9382056c1 service swift] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:28.006996 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:28.007868 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.008150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.008427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] PATH_INFO: `/v3/users/5d74c0a687664ac78d3c413251033e49` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.009626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d74c0a687664ac78d3c413251033e49)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:28.011608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:28.058066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d74c0a687664ac78d3c413251033e49'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.066043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d74c0a687664ac78d3c413251033e49'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.072998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d74c0a687664ac78d3c413251033e49'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.073618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d74c0a687664ac78d3c413251033e49'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.086305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d74c0a687664ac78d3c413251033e49'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.092347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1146/2273] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:52:27 2022] DELETE /identity/v3/users/5d74c0a687664ac78d3c413251033e49 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:28.097316 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab7c99-b171-40de-b867-7696489a5329 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.097778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab7c99-b171-40de-b867-7696489a5329 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.097871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab7c99-b171-40de-b867-7696489a5329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.115716 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-1bab7c99-b171-40de-b867-7696489a5329 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:28.116897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1bab7c99-b171-40de-b867-7696489a5329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:28.146925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:28.178923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1128/2274] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:28.188424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:28.189865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:28.204796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:28.251280 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3d979-44c8-4190-aa15-3824ef0d5186 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:28.255020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1def1e-5ebf-4703-8577-1a6d64bb25c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:28.255662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1def1e-5ebf-4703-8577-1a6d64bb25c4 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.255944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1def1e-5ebf-4703-8577-1a6d64bb25c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.256221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1def1e-5ebf-4703-8577-1a6d64bb25c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.265915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1def1e-5ebf-4703-8577-1a6d64bb25c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:28.267472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1def1e-5ebf-4703-8577-1a6d64bb25c4 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:28.277182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1147/2275] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:52:28.464919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab7c99-b171-40de-b867-7696489a5329 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:28.466467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1bab7c99-b171-40de-b867-7696489a5329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:28.490083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab7c99-b171-40de-b867-7696489a5329 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:28.495455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:28.496318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.496596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.496867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] PATH_INFO: `/v3/projects/718238a5852e4d9a93d85fa8fcf41643` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.498039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=718238a5852e4d9a93d85fa8fcf41643)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:28.499955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:28.550060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718238a5852e4d9a93d85fa8fcf41643', 'request_id': 'req-07286c21-0679-450a-a7a4-9a974bd787f7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.556108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718238a5852e4d9a93d85fa8fcf41643', 'request_id': 'req-07286c21-0679-450a-a7a4-9a974bd787f7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.564162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718238a5852e4d9a93d85fa8fcf41643', 'request_id': 'req-07286c21-0679-450a-a7a4-9a974bd787f7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.574044 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] The token cache is being invalidate because project 718238a5852e4d9a93d85fa8fcf41643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:52:28.574328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.574966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-07286c21-0679-450a-a7a4-9a974bd787f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:28.576495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1129/2276] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:52:28 2022] DELETE /identity/v3/projects/718238a5852e4d9a93d85fa8fcf41643 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:28.582390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ac2cc-c571-46bb-a16e-3dad01c7f738 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.582683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ac2cc-c571-46bb-a16e-3dad01c7f738 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.583647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ac2cc-c571-46bb-a16e-3dad01c7f738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.613693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-525ac2cc-c571-46bb-a16e-3dad01c7f738 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:28.615134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-525ac2cc-c571-46bb-a16e-3dad01c7f738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:28.675248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1148/2277] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:28.682796 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7204fee-f17f-4767-a367-6bc706d2cb98 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.683087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7204fee-f17f-4767-a367-6bc706d2cb98 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.683365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7204fee-f17f-4767-a367-6bc706d2cb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.701331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e7204fee-f17f-4767-a367-6bc706d2cb98 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:28.702938 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e7204fee-f17f-4767-a367-6bc706d2cb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:28.712125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1130/2278] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:28.716389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ac2cc-c571-46bb-a16e-3dad01c7f738 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:28.726677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ac2cc-c571-46bb-a16e-3dad01c7f738 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:28.731226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0fbff-8e02-4313-a4b8-aaa8e0ea2d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:28.731872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0fbff-8e02-4313-a4b8-aaa8e0ea2d8f admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.732154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0fbff-8e02-4313-a4b8-aaa8e0ea2d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.732432 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0fbff-8e02-4313-a4b8-aaa8e0ea2d8f admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.733149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0fbff-8e02-4313-a4b8-aaa8e0ea2d8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:52:28.733457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0fbff-8e02-4313-a4b8-aaa8e0ea2d8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:28.735079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0fbff-8e02-4313-a4b8-aaa8e0ea2d8f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:28.746875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1149/2279] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:52:28.751590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7204fee-f17f-4767-a367-6bc706d2cb98 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:28.760628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7204fee-f17f-4767-a367-6bc706d2cb98 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:28.765139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1657ef-a4fd-49c1-b123-04c8b00f9ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:28.765774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1657ef-a4fd-49c1-b123-04c8b00f9ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.766643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1657ef-a4fd-49c1-b123-04c8b00f9ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.766962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1657ef-a4fd-49c1-b123-04c8b00f9ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.767653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1657ef-a4fd-49c1-b123-04c8b00f9ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:28.769515 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1657ef-a4fd-49c1-b123-04c8b00f9ae7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:28.795914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1131/2280] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:52:28 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:52:28.800696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0fbff-8e02-4313-a4b8-aaa8e0ea2d8f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:28.810597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0fbff-8e02-4313-a4b8-aaa8e0ea2d8f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:28.873187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce0d1-c089-4cab-9780-976bcaccf5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:28.873862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce0d1-c089-4cab-9780-976bcaccf5ad admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.874146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce0d1-c089-4cab-9780-976bcaccf5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.874424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce0d1-c089-4cab-9780-976bcaccf5ad admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.875150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce0d1-c089-4cab-9780-976bcaccf5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:28.877634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce0d1-c089-4cab-9780-976bcaccf5ad admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:28.897036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1150/2281] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:52:28 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:52:28.901153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1657ef-a4fd-49c1-b123-04c8b00f9ae7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:28.914127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1657ef-a4fd-49c1-b123-04c8b00f9ae7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:28.918654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a90e-0275-4958-a9d0-67267a0b468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:28.919347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a90e-0275-4958-a9d0-67267a0b468f admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.919628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a90e-0275-4958-a9d0-67267a0b468f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.919904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a90e-0275-4958-a9d0-67267a0b468f admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.920623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112a90e-0275-4958-a9d0-67267a0b468f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:28.922208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112a90e-0275-4958-a9d0-67267a0b468f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:28.932570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1132/2282] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:52:28.936380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce0d1-c089-4cab-9780-976bcaccf5ad admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:28.947015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce0d1-c089-4cab-9780-976bcaccf5ad admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:28.953425 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43de742-249c-4f36-b799-843643683801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:28.954645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43de742-249c-4f36-b799-843643683801 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.954964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43de742-249c-4f36-b799-843643683801 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.955248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43de742-249c-4f36-b799-843643683801 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:28.955977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43de742-249c-4f36-b799-843643683801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:28.957576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43de742-249c-4f36-b799-843643683801 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:28.967964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1151/2283] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:52:28.972071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a90e-0275-4958-a9d0-67267a0b468f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:28.982279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a90e-0275-4958-a9d0-67267a0b468f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:28.987101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552af9-a9cd-43fa-8b0a-1e00072e13d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:28.987987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552af9-a9cd-43fa-8b0a-1e00072e13d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:28.988270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552af9-a9cd-43fa-8b0a-1e00072e13d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:28.988544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552af9-a9cd-43fa-8b0a-1e00072e13d7 admin admin] PATH_INFO: `/v3/projects/6571620c4d5a473ba4736788b57053d2/users/399411fcb0974846beaf2cad2e48c752/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:29.005030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06552af9-a9cd-43fa-8b0a-1e00072e13d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6571620c4d5a473ba4736788b57053d2, user_id=399411fcb0974846beaf2cad2e48c752, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:29.007714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06552af9-a9cd-43fa-8b0a-1e00072e13d7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:29.021889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1133/2284] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:52:28 2022] PUT /identity/v3/projects/6571620c4d5a473ba4736788b57053d2/users/399411fcb0974846beaf2cad2e48c752/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:29.032318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2fb2c-8324-4be6-8f9f-8aedbc9e24dd None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:29.032607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2fb2c-8324-4be6-8f9f-8aedbc9e24dd None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:29.032888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2fb2c-8324-4be6-8f9f-8aedbc9e24dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:29.058656 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:29.062617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-89b2fb2c-8324-4be6-8f9f-8aedbc9e24dd None None] MFA Rules not processed for user `399411fcb0974846beaf2cad2e48c752`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:29.064773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-89b2fb2c-8324-4be6-8f9f-8aedbc9e24dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:29.137812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1152/2285] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:52:29.148439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552af9-a9cd-43fa-8b0a-1e00072e13d7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:29.150318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-06552af9-a9cd-43fa-8b0a-1e00072e13d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:52:29.210881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552af9-a9cd-43fa-8b0a-1e00072e13d7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:29.214560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f178e4a-e3d4-43a7-9347-7d975d9fd921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:29.215241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f178e4a-e3d4-43a7-9347-7d975d9fd921 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:29.215527 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f178e4a-e3d4-43a7-9347-7d975d9fd921 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:29.215808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f178e4a-e3d4-43a7-9347-7d975d9fd921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:29.224532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:29.225720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f178e4a-e3d4-43a7-9347-7d975d9fd921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:29.227715 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f178e4a-e3d4-43a7-9347-7d975d9fd921 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:29.240828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1134/2286] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:52:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:52:39.134249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=26 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:39.241732 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:52:59.779005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2fb2c-8324-4be6-8f9f-8aedbc9e24dd None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:52:59.842710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2fb2c-8324-4be6-8f9f-8aedbc9e24dd None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:52:59.847214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:52:59.848081 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:59.848359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:59.848631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] PATH_INFO: `/v3/users/399411fcb0974846beaf2cad2e48c752` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:59.850380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=399411fcb0974846beaf2cad2e48c752)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:52:59.852321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:52:59.898612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '399411fcb0974846beaf2cad2e48c752'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:59.906583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '399411fcb0974846beaf2cad2e48c752'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:59.914598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '399411fcb0974846beaf2cad2e48c752'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:59.915260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '399411fcb0974846beaf2cad2e48c752'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:59.927599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '399411fcb0974846beaf2cad2e48c752'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:52:59.933929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1153/2287] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:52:59 2022] DELETE /identity/v3/users/399411fcb0974846beaf2cad2e48c752 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:52:59.938418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4faa9-7e18-4667-9467-1905ddf6d979 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:52:59.938931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4faa9-7e18-4667-9467-1905ddf6d979 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:52:59.939039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4faa9-7e18-4667-9467-1905ddf6d979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:52:59.956643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-5fa4faa9-7e18-4667-9467-1905ddf6d979 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:52:59.957820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5fa4faa9-7e18-4667-9467-1905ddf6d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:53:00.019766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1135/2288] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:53:00.029860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:53:00.031292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:53:00.088033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d69ff-338c-4aaa-98cb-82d647a439a8 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:53:00.091727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da54a0-dfb7-4835-b153-63b8a55a06f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:53:00.092443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da54a0-dfb7-4835-b153-63b8a55a06f2 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:53:00.092711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da54a0-dfb7-4835-b153-63b8a55a06f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:53:00.092993 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da54a0-dfb7-4835-b153-63b8a55a06f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:53:00.102474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da54a0-dfb7-4835-b153-63b8a55a06f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:53:00.103945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da54a0-dfb7-4835-b153-63b8a55a06f2 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:53:00.113295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1154/2289] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:53:00.376814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4faa9-7e18-4667-9467-1905ddf6d979 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:53:00.378331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5fa4faa9-7e18-4667-9467-1905ddf6d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:53:00.391786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4faa9-7e18-4667-9467-1905ddf6d979 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:53:00.398090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:53:00.398899 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:53:00.399304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:53:00.399455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] PATH_INFO: `/v3/projects/6571620c4d5a473ba4736788b57053d2` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:53:00.400726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6571620c4d5a473ba4736788b57053d2)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:53:00.402538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:53:00.453923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6571620c4d5a473ba4736788b57053d2', 'request_id': 'req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:53:00.460823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6571620c4d5a473ba4736788b57053d2', 'request_id': 'req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:53:00.467291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6571620c4d5a473ba4736788b57053d2', 'request_id': 'req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:53:00.477580 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] The token cache is being invalidate because project 6571620c4d5a473ba4736788b57053d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:53:00.477870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:53:00.478486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8e31eef1-5eb7-4bc3-bf8b-e1062ecbbd4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:53:00.480059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1136/2290] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:53:00 2022] DELETE /identity/v3/projects/6571620c4d5a473ba4736788b57053d2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:53:10.109214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:53:10.474743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:53:41.259250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da54a0-dfb7-4835-b153-63b8a55a06f2 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:53:41.260848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-96da54a0-dfb7-4835-b153-63b8a55a06f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:53:41.351621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da54a0-dfb7-4835-b153-63b8a55a06f2 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:53:41.408990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd34535-bc4e-4be9-b973-57cef1265108 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:53:41.409637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34535-bc4e-4be9-b973-57cef1265108 service placement] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:53:41.409920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34535-bc4e-4be9-b973-57cef1265108 service placement] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:53:41.410198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34535-bc4e-4be9-b973-57cef1265108 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:53:41.412263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ecd34535-bc4e-4be9-b973-57cef1265108 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:53:41.423284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd34535-bc4e-4be9-b973-57cef1265108 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:53:41.424860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd34535-bc4e-4be9-b973-57cef1265108 service placement] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:53:41.435093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1155/2291] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:53:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:53:43.486426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7475b1d-7e03-4e1a-ac5d-2a16fe03c395 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:53:43.487301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7475b1d-7e03-4e1a-ac5d-2a16fe03c395 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:53:43.487835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7475b1d-7e03-4e1a-ac5d-2a16fe03c395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:53:43.504901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b7475b1d-7e03-4e1a-ac5d-2a16fe03c395 None None] MFA Rules not processed for user `f8fcb09990db433e8ac0af3de37bdbc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:53:43.506101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b7475b1d-7e03-4e1a-ac5d-2a16fe03c395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:53:43.558611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1137/2292] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:53:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:53:51.430391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=67 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:53:53.552201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=25 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:53:55.008106 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde00059-6715-4675-ab32-004459714e42 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:53:55.008581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde00059-6715-4675-ab32-004459714e42 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:53:55.008691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde00059-6715-4675-ab32-004459714e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:53:55.025378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-cde00059-6715-4675-ab32-004459714e42 None None] MFA Rules not processed for user `e872bc2e7b214f0889ba95cc9352c59b`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:53:55.026604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cde00059-6715-4675-ab32-004459714e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:53:55.080134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1156/2293] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:53:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:54:05.073549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=25 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:54:14.008233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2117b-029d-4f1f-966f-f90de367fca6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:54:14.008753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2117b-029d-4f1f-966f-f90de367fca6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:54:14.008860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2117b-029d-4f1f-966f-f90de367fca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:54:14.026032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-87e2117b-029d-4f1f-966f-f90de367fca6 None None] MFA Rules not processed for user `fb838f931b7748e2baf6c2315d3cbd75`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:54:14.027286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-87e2117b-029d-4f1f-966f-f90de367fca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:54:14.080771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1138/2294] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:54:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:54:24.074833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=25 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:54:48.872710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42269c0-d528-419a-8dee-b07aa1bec72f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:54:48.873430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42269c0-d528-419a-8dee-b07aa1bec72f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:54:48.873430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42269c0-d528-419a-8dee-b07aa1bec72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:54:48.933573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b42269c0-d528-419a-8dee-b07aa1bec72f None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:54:48.934822 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b42269c0-d528-419a-8dee-b07aa1bec72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:54:49.006036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1157/2295] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:54:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:54:49.059742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2117b-029d-4f1f-966f-f90de367fca6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:54:49.061142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-87e2117b-029d-4f1f-966f-f90de367fca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:54:49.155015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2117b-029d-4f1f-966f-f90de367fca6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:54:49.160468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e683f89-e352-4aeb-8dea-94119bb396fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:54:49.161222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e683f89-e352-4aeb-8dea-94119bb396fc service swift] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:54:49.161833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e683f89-e352-4aeb-8dea-94119bb396fc service swift] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:54:49.162303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e683f89-e352-4aeb-8dea-94119bb396fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:54:49.177083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e683f89-e352-4aeb-8dea-94119bb396fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:54:49.177083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e683f89-e352-4aeb-8dea-94119bb396fc service swift] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:54:49.182468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1139/2296] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:54:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:54:53.819151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42269c0-d528-419a-8dee-b07aa1bec72f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:54:53.820780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b42269c0-d528-419a-8dee-b07aa1bec72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:54:53.898469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42269c0-d528-419a-8dee-b07aa1bec72f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:54:53.902490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:54:53.903183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:54:53.903464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:54:53.903742 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:54:53.905644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:54:53.996928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:54:54.015727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:54:54.017368 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:54:54.045102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1158/2297] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:54:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:54:55.265965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e683f89-e352-4aeb-8dea-94119bb396fc service swift] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:54:55.275816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e683f89-e352-4aeb-8dea-94119bb396fc service swift] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:54:55.279562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:54:55.280349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:54:55.280577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:54:55.280848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:54:55.282822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:54:55.300790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:54:55.302997 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:54:55.316258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1140/2298] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:54:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:03.628877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:03.630618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:03.644274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba397bc-cbf7-4f56-9277-8dce7f2e14dc service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:03.650243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9406e-c234-40ea-a55a-c88ced3eb8aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:03.650916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9406e-c234-40ea-a55a-c88ced3eb8aa service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:03.651203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9406e-c234-40ea-a55a-c88ced3eb8aa service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:03.651486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9406e-c234-40ea-a55a-c88ced3eb8aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:03.653208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-41c9406e-c234-40ea-a55a-c88ced3eb8aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:03.692146 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:03.693674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:03.715390 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9406e-c234-40ea-a55a-c88ced3eb8aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:03.716988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9406e-c234-40ea-a55a-c88ced3eb8aa service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:03.726784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1159/2299] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:03.772734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623d0da-5dc8-49f9-aab8-bf62ef63f6a7 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:03.776442 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b78cf-a6e5-42cf-b17e-da4dace2accc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:03.777089 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b78cf-a6e5-42cf-b17e-da4dace2accc service glance] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:03.777372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b78cf-a6e5-42cf-b17e-da4dace2accc service glance] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:03.777667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b78cf-a6e5-42cf-b17e-da4dace2accc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:03.787603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b78cf-a6e5-42cf-b17e-da4dace2accc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:03.789061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b78cf-a6e5-42cf-b17e-da4dace2accc service glance] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:03.798496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1141/2300] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:10.284884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9406e-c234-40ea-a55a-c88ced3eb8aa service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:10.295056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9406e-c234-40ea-a55a-c88ced3eb8aa service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:10.298705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d012e-4e72-4938-a657-243416dd4aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:10.299395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d012e-4e72-4938-a657-243416dd4aac service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.299677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d012e-4e72-4938-a657-243416dd4aac service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.299957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d012e-4e72-4938-a657-243416dd4aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.302281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-348d012e-4e72-4938-a657-243416dd4aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:10.311774 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d012e-4e72-4938-a657-243416dd4aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:10.313195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d012e-4e72-4938-a657-243416dd4aac service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:10.321640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1160/2301] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:10.423513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bffb66-fb80-44c4-afa5-f69a3170c99b None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.424106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bffb66-fb80-44c4-afa5-f69a3170c99b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.424106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bffb66-fb80-44c4-afa5-f69a3170c99b None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.425609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1142/2302] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:55:10.432852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587e3ac-8473-49c6-ae6f-aabdbb662cc4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.433194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587e3ac-8473-49c6-ae6f-aabdbb662cc4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.433446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587e3ac-8473-49c6-ae6f-aabdbb662cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.476219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5587e3ac-8473-49c6-ae6f-aabdbb662cc4 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:55:10.477559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5587e3ac-8473-49c6-ae6f-aabdbb662cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:10.488182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1161/2303] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:55:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:55:10.500070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bffb66-fb80-44c4-afa5-f69a3170c99b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:10.502058 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-50bffb66-fb80-44c4-afa5-f69a3170c99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:10.518522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bffb66-fb80-44c4-afa5-f69a3170c99b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:10.522327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3375b-2532-4c0b-8569-6bc7e5f86a0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:10.523074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3375b-2532-4c0b-8569-6bc7e5f86a0a service glance] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.523447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3375b-2532-4c0b-8569-6bc7e5f86a0a service glance] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.523734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3375b-2532-4c0b-8569-6bc7e5f86a0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.534714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3375b-2532-4c0b-8569-6bc7e5f86a0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:10.537149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3375b-2532-4c0b-8569-6bc7e5f86a0a service glance] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:10.548215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1143/2304] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:10.614132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cd0b6-8cec-4eee-a350-45fd5f407056 None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.614419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cd0b6-8cec-4eee-a350-45fd5f407056 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.614699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cd0b6-8cec-4eee-a350-45fd5f407056 None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.616091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1162/2305] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:55:10.620821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c338ee-100c-4921-a24c-e5be378002f3 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.621180 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c338ee-100c-4921-a24c-e5be378002f3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.621484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c338ee-100c-4921-a24c-e5be378002f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.640952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d4c338ee-100c-4921-a24c-e5be378002f3 None None] MFA Rules not processed for user `bf1915b9919442d692bcedf25f53320c`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:55:10.642138 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d4c338ee-100c-4921-a24c-e5be378002f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:10.649301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1144/2306] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:55:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:55:10.661970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cd0b6-8cec-4eee-a350-45fd5f407056 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:10.663916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a59cd0b6-8cec-4eee-a350-45fd5f407056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:10.677885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cd0b6-8cec-4eee-a350-45fd5f407056 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:10.681446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3065e-6c82-4ec2-9048-47374a542b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:10.682083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3065e-6c82-4ec2-9048-47374a542b98 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.682378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3065e-6c82-4ec2-9048-47374a542b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.682800 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3065e-6c82-4ec2-9048-47374a542b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.690308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3065e-6c82-4ec2-9048-47374a542b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:10.691857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3065e-6c82-4ec2-9048-47374a542b98 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:10.700075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1163/2307] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:10.787525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc566722-2164-460e-9565-722bab350945 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.787815 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc566722-2164-460e-9565-722bab350945 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.788093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc566722-2164-460e-9565-722bab350945 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.789463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1145/2308] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:55:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:55:10.795718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d958d9f-b9b2-4dce-9447-cec5a9a0961d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.795718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d958d9f-b9b2-4dce-9447-cec5a9a0961d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.796154 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d958d9f-b9b2-4dce-9447-cec5a9a0961d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.816035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-8d958d9f-b9b2-4dce-9447-cec5a9a0961d None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:55:10.817533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8d958d9f-b9b2-4dce-9447-cec5a9a0961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:10.826921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1164/2309] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:55:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:55:10.837168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc566722-2164-460e-9565-722bab350945 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:10.848001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc566722-2164-460e-9565-722bab350945 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:10.852050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437cd9d-fa70-4803-a752-c39bd5b8b68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:10.852681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437cd9d-fa70-4803-a752-c39bd5b8b68d service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.852961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437cd9d-fa70-4803-a752-c39bd5b8b68d service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.853239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437cd9d-fa70-4803-a752-c39bd5b8b68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.863054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437cd9d-fa70-4803-a752-c39bd5b8b68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:10.865025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437cd9d-fa70-4803-a752-c39bd5b8b68d service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:10.875369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1146/2310] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:10.938384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d958d9f-b9b2-4dce-9447-cec5a9a0961d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:10.939810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8d958d9f-b9b2-4dce-9447-cec5a9a0961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:10.953318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d958d9f-b9b2-4dce-9447-cec5a9a0961d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:10.958796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1c432-55c3-4b26-933b-ddb0d4c490a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:10.959422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1c432-55c3-4b26-933b-ddb0d4c490a6 service placement] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:10.959709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1c432-55c3-4b26-933b-ddb0d4c490a6 service placement] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:10.959981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1c432-55c3-4b26-933b-ddb0d4c490a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:10.961675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bab1c432-55c3-4b26-933b-ddb0d4c490a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:10.972460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1c432-55c3-4b26-933b-ddb0d4c490a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:10.974646 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1c432-55c3-4b26-933b-ddb0d4c490a6 service placement] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:10.984225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1165/2311] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:12.669925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437cd9d-fa70-4803-a752-c39bd5b8b68d service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:12.678910 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437cd9d-fa70-4803-a752-c39bd5b8b68d service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:12.682476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8894e-e800-4cd4-befb-b1a0dcbed0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:12.683151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8894e-e800-4cd4-befb-b1a0dcbed0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:12.683438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8894e-e800-4cd4-befb-b1a0dcbed0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:12.683718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8894e-e800-4cd4-befb-b1a0dcbed0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:12.685473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-04a8894e-e800-4cd4-befb-b1a0dcbed0d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:12.691470 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:55:12.696610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8894e-e800-4cd4-befb-b1a0dcbed0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:12.698023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8894e-e800-4cd4-befb-b1a0dcbed0d2 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:12.705907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1147/2312] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:13.545336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5778f-6347-494a-959f-7464bdaf0bf3 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:13.545790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5778f-6347-494a-959f-7464bdaf0bf3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:13.545921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5778f-6347-494a-959f-7464bdaf0bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:13.562954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-59e5778f-6347-494a-959f-7464bdaf0bf3 None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:55:13.564129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-59e5778f-6347-494a-959f-7464bdaf0bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:13.572022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1166/2313] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:55:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:55:13.584656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8894e-e800-4cd4-befb-b1a0dcbed0d2 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:13.611289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8894e-e800-4cd4-befb-b1a0dcbed0d2 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:13.618335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10d3ad-1a15-4bb0-b123-b5c0cd0a4f72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:13.619033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d3ad-1a15-4bb0-b123-b5c0cd0a4f72 service swift] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:13.619319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d3ad-1a15-4bb0-b123-b5c0cd0a4f72 service swift] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:13.619623 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d3ad-1a15-4bb0-b123-b5c0cd0a4f72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:13.628555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10d3ad-1a15-4bb0-b123-b5c0cd0a4f72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:13.630648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10d3ad-1a15-4bb0-b123-b5c0cd0a4f72 service swift] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:13.639084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1148/2314] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:55:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:55:15.331978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d0748-ebe1-4bec-99aa-28890c6f4fbf None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:15.333146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d0748-ebe1-4bec-99aa-28890c6f4fbf None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:15.333242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d0748-ebe1-4bec-99aa-28890c6f4fbf None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:15.334770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1167/2315] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:55:15.350760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9c4ee-5d2c-414c-9311-51ee84643e06 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:15.351196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9c4ee-5d2c-414c-9311-51ee84643e06 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:15.351325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9c4ee-5d2c-414c-9311-51ee84643e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:15.406600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f1d9c4ee-5d2c-414c-9311-51ee84643e06 None None] MFA Rules not processed for user `a786a95c48c348dbba269e0e33265271`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:55:15.408771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f1d9c4ee-5d2c-414c-9311-51ee84643e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:15.419239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1149/2316] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:55:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:55:15.648316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d0748-ebe1-4bec-99aa-28890c6f4fbf None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:15.671594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d0748-ebe1-4bec-99aa-28890c6f4fbf None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:15.690845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cab23-fd67-4a3a-a30e-7eccb60bb886 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:15.692470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cab23-fd67-4a3a-a30e-7eccb60bb886 service placement] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:15.692953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cab23-fd67-4a3a-a30e-7eccb60bb886 service placement] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:15.693560 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cab23-fd67-4a3a-a30e-7eccb60bb886 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:15.704742 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9c4ee-5d2c-414c-9311-51ee84643e06 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:15.706214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f1d9c4ee-5d2c-414c-9311-51ee84643e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:15.710426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cab23-fd67-4a3a-a30e-7eccb60bb886 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:15.712073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cab23-fd67-4a3a-a30e-7eccb60bb886 service placement] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:15.740884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1168/2317] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:15.791056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9c4ee-5d2c-414c-9311-51ee84643e06 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:15.795683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:15.796591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:15.796876 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:15.797162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] PATH_INFO: `/v3/users/e872bc2e7b214f0889ba95cc9352c59b` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:15.798411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e872bc2e7b214f0889ba95cc9352c59b)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:15.800402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:15.851535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e872bc2e7b214f0889ba95cc9352c59b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:15.858966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e872bc2e7b214f0889ba95cc9352c59b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:15.867353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e872bc2e7b214f0889ba95cc9352c59b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:15.868070 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e872bc2e7b214f0889ba95cc9352c59b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:15.868691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e872bc2e7b214f0889ba95cc9352c59b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:15.907021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1150/2318] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:55:15 2022] DELETE /identity/v3/users/e872bc2e7b214f0889ba95cc9352c59b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:55:16.094828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cab23-fd67-4a3a-a30e-7eccb60bb886 service placement] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:16.106832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cab23-fd67-4a3a-a30e-7eccb60bb886 service placement] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:16.158902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:55:16.169495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:16.170352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:16.170628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:16.170941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] PATH_INFO: `/v3/projects/bf53255e905344409fd64e4fba710dbc` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:16.172122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf53255e905344409fd64e4fba710dbc)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:16.174025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:16.230633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf53255e905344409fd64e4fba710dbc', 'request_id': 'req-5d3e48fa-cba9-4c69-af34-c3a6498158f4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:16.237979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf53255e905344409fd64e4fba710dbc', 'request_id': 'req-5d3e48fa-cba9-4c69-af34-c3a6498158f4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:16.247179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf53255e905344409fd64e4fba710dbc', 'request_id': 'req-5d3e48fa-cba9-4c69-af34-c3a6498158f4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:16.258085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] The token cache is being invalidate because project bf53255e905344409fd64e4fba710dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:55:16.258372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:16.259047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5d3e48fa-cba9-4c69-af34-c3a6498158f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:55:16.260614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1169/2319] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:55:16 2022] DELETE /identity/v3/projects/bf53255e905344409fd64e4fba710dbc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:55:25.898933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:55:26.254822 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:55:40.929962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:40.931468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:41.005317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345d943-ae55-410e-bde3-cdcf7accf13a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:41.067577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:41.068130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:41.068415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:41.068692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:41.070482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:41.082004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:41.084250 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:41.094471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1151/2320] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:55:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:55:44.695491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332996-58a5-4665-b298-770900f5f9a2 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:44.695951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332996-58a5-4665-b298-770900f5f9a2 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:44.696057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332996-58a5-4665-b298-770900f5f9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:44.714370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-8a332996-58a5-4665-b298-770900f5f9a2 None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:55:44.715510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8a332996-58a5-4665-b298-770900f5f9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:44.768263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1170/2321] 10.222.0.62 () {58 vars in 967 bytes} [Wed Jun 15 15:55:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:55:44.780497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:55:44.781928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:55:44.852944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725da7f9-de60-4900-b2fc-d859603baaa6 service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:55:44.857838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f860442-4225-4767-a63e-6bb605f5233c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:55:44.858490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f860442-4225-4767-a63e-6bb605f5233c service swift] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:55:44.858805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f860442-4225-4767-a63e-6bb605f5233c service swift] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:55:44.859085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f860442-4225-4767-a63e-6bb605f5233c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:55:44.867765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f860442-4225-4767-a63e-6bb605f5233c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:55:44.869360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f860442-4225-4767-a63e-6bb605f5233c service swift] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:55:44.877544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1152/2322] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:55:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:55:54.761905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=25 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:55:54.874213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=94 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:56:02.770595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a332996-58a5-4665-b298-770900f5f9a2 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:56:02.772192 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8a332996-58a5-4665-b298-770900f5f9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:02.788255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a332996-58a5-4665-b298-770900f5f9a2 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:56:02.793689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2ae66-0139-46a4-b0be-e467d8dccd58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:56:02.794327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2ae66-0139-46a4-b0be-e467d8dccd58 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:56:02.794612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2ae66-0139-46a4-b0be-e467d8dccd58 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:56:02.794922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2ae66-0139-46a4-b0be-e467d8dccd58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:56:02.796626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1ae2ae66-0139-46a4-b0be-e467d8dccd58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:02.853757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2ae66-0139-46a4-b0be-e467d8dccd58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:56:02.855389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2ae66-0139-46a4-b0be-e467d8dccd58 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:56:02.864973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1171/2323] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:56:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:56:12.860175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=28 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:56:27.077785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f860442-4225-4767-a63e-6bb605f5233c service swift] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:56:27.089509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f860442-4225-4767-a63e-6bb605f5233c service swift] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:56:27.095831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ab4e3-28d9-4c9b-b541-b3890bfdd34a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:56:27.096477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ab4e3-28d9-4c9b-b541-b3890bfdd34a service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:56:27.096763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ab4e3-28d9-4c9b-b541-b3890bfdd34a service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:56:27.097046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ab4e3-28d9-4c9b-b541-b3890bfdd34a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:56:27.098868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ac7ab4e3-28d9-4c9b-b541-b3890bfdd34a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:27.158024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ab4e3-28d9-4c9b-b541-b3890bfdd34a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:56:27.159638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ab4e3-28d9-4c9b-b541-b3890bfdd34a service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:56:27.168765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1153/2324] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:56:30.358905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2ae66-0139-46a4-b0be-e467d8dccd58 service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:56:30.371691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2ae66-0139-46a4-b0be-e467d8dccd58 service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:56:30.377605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:56:30.378333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:56:30.378687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:56:30.379066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:56:30.380883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:30.394093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:56:30.395732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:56:30.407139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1172/2325] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:56:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:56:34.111013 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19853145-872c-416b-892c-ba00283b8be7 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:56:34.111532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19853145-872c-416b-892c-ba00283b8be7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:56:34.111645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19853145-872c-416b-892c-ba00283b8be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:56:34.128629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-19853145-872c-416b-892c-ba00283b8be7 None None] MFA Rules not processed for user `f8fcb09990db433e8ac0af3de37bdbc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:56:34.129806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-19853145-872c-416b-892c-ba00283b8be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:34.183474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1154/2326] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:56:37.902897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:56:37.904413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:37.918432 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adc1bc-667f-4eba-87c5-ada5a874482e service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:56:37.924950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a9a2b-a3e5-4fe6-b4fd-47dd29e6924a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:56:37.925654 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a9a2b-a3e5-4fe6-b4fd-47dd29e6924a service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:56:37.925971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a9a2b-a3e5-4fe6-b4fd-47dd29e6924a service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:56:37.926190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a9a2b-a3e5-4fe6-b4fd-47dd29e6924a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:56:37.929576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1d1a9a2b-a3e5-4fe6-b4fd-47dd29e6924a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:37.940453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a9a2b-a3e5-4fe6-b4fd-47dd29e6924a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:56:37.942025 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a9a2b-a3e5-4fe6-b4fd-47dd29e6924a service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:56:37.951401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1173/2327] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:56:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:56:40.533941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19853145-872c-416b-892c-ba00283b8be7 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:56:40.547407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19853145-872c-416b-892c-ba00283b8be7 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:56:40.552980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:56:40.553639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:56:40.553923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:56:40.554197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:56:40.556596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:40.569125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:56:40.570693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:56:40.582233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1155/2328] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:56:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:56:45.617983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a892f9-0b7f-4988-b798-7d25ce73e35e None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:56:45.619061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a892f9-0b7f-4988-b798-7d25ce73e35e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:56:45.619178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a892f9-0b7f-4988-b798-7d25ce73e35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:56:45.636288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-18a892f9-0b7f-4988-b798-7d25ce73e35e None None] MFA Rules not processed for user `fb838f931b7748e2baf6c2315d3cbd75`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:56:45.637503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-18a892f9-0b7f-4988-b798-7d25ce73e35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:45.644300 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1174/2329] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:56:45.938514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:56:45.939956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:45.996664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08acef-4528-49c6-adec-cb3b37dabd9c service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:56:46.000275 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4872a8-e6ef-4e78-a989-bddf79ea5189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:56:46.000911 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4872a8-e6ef-4e78-a989-bddf79ea5189 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:56:46.001194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4872a8-e6ef-4e78-a989-bddf79ea5189 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:56:46.001471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4872a8-e6ef-4e78-a989-bddf79ea5189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:56:46.003273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cb4872a8-e6ef-4e78-a989-bddf79ea5189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:56:46.012894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4872a8-e6ef-4e78-a989-bddf79ea5189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:56:46.014231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4872a8-e6ef-4e78-a989-bddf79ea5189 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:56:46.021903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1156/2330] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:56:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:56:55.630946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=15 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:56:56.018534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=85 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:57:05.926386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead5a01-5167-4086-9c08-00c5b2f52b6f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:57:05.926951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead5a01-5167-4086-9c08-00c5b2f52b6f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:57:05.926951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead5a01-5167-4086-9c08-00c5b2f52b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:57:05.944349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-1ead5a01-5167-4086-9c08-00c5b2f52b6f None None] MFA Rules not processed for user `3af6df3efbb74db4958654503e17d82e`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:57:05.946178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1ead5a01-5167-4086-9c08-00c5b2f52b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:57:05.999573 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1175/2331] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:57:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:57:15.992389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=25 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:57:46.376101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468d8ea-0148-42ac-a3ad-feb54517aca1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:57:46.376641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468d8ea-0148-42ac-a3ad-feb54517aca1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:57:46.376641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468d8ea-0148-42ac-a3ad-feb54517aca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:57:46.394099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-a468d8ea-0148-42ac-a3ad-feb54517aca1 None None] MFA Rules not processed for user `434c81b82dee404f8d143f1bdf56d3ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:57:46.395316 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a468d8ea-0148-42ac-a3ad-feb54517aca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:57:46.402930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1157/2332] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:57:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:57:56.389057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:19.610213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead5a01-5167-4086-9c08-00c5b2f52b6f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:19.611609 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1ead5a01-5167-4086-9c08-00c5b2f52b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:19.623351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead5a01-5167-4086-9c08-00c5b2f52b6f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:19.626848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:19.627481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:19.627757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:19.628029 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:19.629710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:19.692418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:19.693867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:19.704111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1176/2333] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:22.818688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468d8ea-0148-42ac-a3ad-feb54517aca1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:22.820144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a468d8ea-0148-42ac-a3ad-feb54517aca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:22.875334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:22.878476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:22.896607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840cc66-627f-4fc6-b77b-bedfa75c726a service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:22.899244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468d8ea-0148-42ac-a3ad-feb54517aca1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:22.901450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:22.902323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:22.902608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:22.902936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] PATH_INFO: `/v3/users/f8fcb09990db433e8ac0af3de37bdbc7` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:22.904119 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8fcb09990db433e8ac0af3de37bdbc7)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:22.905718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70669982-a11e-433b-855a-599434bf989c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:22.906077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:22.906362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70669982-a11e-433b-855a-599434bf989c service placement] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:22.906644 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70669982-a11e-433b-855a-599434bf989c service placement] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:22.907025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70669982-a11e-433b-855a-599434bf989c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:22.909672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-70669982-a11e-433b-855a-599434bf989c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:22.922414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70669982-a11e-433b-855a-599434bf989c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:22.924065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70669982-a11e-433b-855a-599434bf989c service placement] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:22.936990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1158/2334] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:22.959692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8fcb09990db433e8ac0af3de37bdbc7'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:22.969357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8fcb09990db433e8ac0af3de37bdbc7'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:22.977608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8fcb09990db433e8ac0af3de37bdbc7'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:22.978324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8fcb09990db433e8ac0af3de37bdbc7'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:22.991487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca9dfa5c-7d12-43a2-9595-0f27e7d5a48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8fcb09990db433e8ac0af3de37bdbc7'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:22.999236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1177/2335] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:58:22 2022] DELETE /identity/v3/users/f8fcb09990db433e8ac0af3de37bdbc7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:58:23.167490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70669982-a11e-433b-855a-599434bf989c service placement] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:23.178863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70669982-a11e-433b-855a-599434bf989c service placement] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:23.244722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:23.245583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.245870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.246150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] PATH_INFO: `/v3/projects/f2ed62f01ef846569c069e52fcfee36a` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.247353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2ed62f01ef846569c069e52fcfee36a)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:23.249297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:23.294805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2ed62f01ef846569c069e52fcfee36a', 'request_id': 'req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:23.300780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2ed62f01ef846569c069e52fcfee36a', 'request_id': 'req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:23.307725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2ed62f01ef846569c069e52fcfee36a', 'request_id': 'req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:23.318327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] The token cache is being invalidate because project f2ed62f01ef846569c069e52fcfee36a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:58:23.318626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:23.319268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3bcfa5bf-189a-40dd-b8f8-7ace54fe0aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:23.320782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1159/2336] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:58:23 2022] DELETE /identity/v3/projects/f2ed62f01ef846569c069e52fcfee36a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:23.326318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2e7c4-3763-45e0-9f71-56eb6f811c3a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.326606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2e7c4-3763-45e0-9f71-56eb6f811c3a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.326934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2e7c4-3763-45e0-9f71-56eb6f811c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.345011 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-86a2e7c4-3763-45e0-9f71-56eb6f811c3a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:23.346220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-86a2e7c4-3763-45e0-9f71-56eb6f811c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:23.404802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1178/2337] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:23.412690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59420f63-a3ae-42ee-a091-1ae4f8360cf1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.412986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59420f63-a3ae-42ee-a091-1ae4f8360cf1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.413269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59420f63-a3ae-42ee-a091-1ae4f8360cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.431147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-59420f63-a3ae-42ee-a091-1ae4f8360cf1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:23.432394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-59420f63-a3ae-42ee-a091-1ae4f8360cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:23.444147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1160/2338] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:23.448424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2e7c4-3763-45e0-9f71-56eb6f811c3a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:23.458750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2e7c4-3763-45e0-9f71-56eb6f811c3a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:23.463464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf02e11-6551-47d8-8b25-9f604bc015a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:23.464112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02e11-6551-47d8-8b25-9f604bc015a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.464395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02e11-6551-47d8-8b25-9f604bc015a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.464674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02e11-6551-47d8-8b25-9f604bc015a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.466004 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf02e11-6551-47d8-8b25-9f604bc015a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:58:23.466312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf02e11-6551-47d8-8b25-9f604bc015a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:23.467948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf02e11-6551-47d8-8b25-9f604bc015a6 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:23.469423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:23.481955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1179/2339] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:58:23.486147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59420f63-a3ae-42ee-a091-1ae4f8360cf1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:23.495285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59420f63-a3ae-42ee-a091-1ae4f8360cf1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:23.499918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6b921-5c7e-4a54-88c8-a255285ecfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:23.500560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6b921-5c7e-4a54-88c8-a255285ecfbc admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.500840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6b921-5c7e-4a54-88c8-a255285ecfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.501115 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6b921-5c7e-4a54-88c8-a255285ecfbc admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.501838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6b921-5c7e-4a54-88c8-a255285ecfbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:23.503752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6b921-5c7e-4a54-88c8-a255285ecfbc admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:23.513818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:23.542447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1161/2340] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:58:23 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:58:23.546626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf02e11-6551-47d8-8b25-9f604bc015a6 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:23.560984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf02e11-6551-47d8-8b25-9f604bc015a6 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:23.616971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665da506-990a-4364-8c60-397ae580c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:23.617637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665da506-990a-4364-8c60-397ae580c4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.617924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665da506-990a-4364-8c60-397ae580c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.618202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665da506-990a-4364-8c60-397ae580c4a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.618928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665da506-990a-4364-8c60-397ae580c4a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:23.620835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665da506-990a-4364-8c60-397ae580c4a7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:23.640698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1180/2341] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:58:23 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:58:23.644799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6b921-5c7e-4a54-88c8-a255285ecfbc admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:23.655660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6b921-5c7e-4a54-88c8-a255285ecfbc admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:23.660407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23bae4-2331-4ee2-b714-dea2d555983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:23.661058 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23bae4-2331-4ee2-b714-dea2d555983d admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.661334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23bae4-2331-4ee2-b714-dea2d555983d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.661607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23bae4-2331-4ee2-b714-dea2d555983d admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.662323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23bae4-2331-4ee2-b714-dea2d555983d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:23.664125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23bae4-2331-4ee2-b714-dea2d555983d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:23.673821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1162/2342] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:58:23.677673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665da506-990a-4364-8c60-397ae580c4a7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:23.688229 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665da506-990a-4364-8c60-397ae580c4a7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:23.692874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e5769-c5b9-4ded-bbe6-c057df2a6871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:23.693524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e5769-c5b9-4ded-bbe6-c057df2a6871 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.693809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e5769-c5b9-4ded-bbe6-c057df2a6871 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.694089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e5769-c5b9-4ded-bbe6-c057df2a6871 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.694844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e5769-c5b9-4ded-bbe6-c057df2a6871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:23.696449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e5769-c5b9-4ded-bbe6-c057df2a6871 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:23.707050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1181/2343] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:23.711584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23bae4-2331-4ee2-b714-dea2d555983d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:23.723423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23bae4-2331-4ee2-b714-dea2d555983d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:23.728077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea1971-a41e-4b24-bb14-f6d52f7d73e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:23.728983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea1971-a41e-4b24-bb14-f6d52f7d73e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.729266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea1971-a41e-4b24-bb14-f6d52f7d73e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.729543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea1971-a41e-4b24-bb14-f6d52f7d73e4 admin admin] PATH_INFO: `/v3/projects/117dafee744049e38fa222d261381e71/users/5fb568fb6097445ea9c04eb33bbd48fd/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.744328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea1971-a41e-4b24-bb14-f6d52f7d73e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=117dafee744049e38fa222d261381e71, user_id=5fb568fb6097445ea9c04eb33bbd48fd, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:23.747675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea1971-a41e-4b24-bb14-f6d52f7d73e4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:23.760607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1163/2344] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:58:23 2022] PUT /identity/v3/projects/117dafee744049e38fa222d261381e71/users/5fb568fb6097445ea9c04eb33bbd48fd/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:23.772127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd92438-e764-453c-ac6b-70bdc36a44eb None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.772423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd92438-e764-453c-ac6b-70bdc36a44eb None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.772705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd92438-e764-453c-ac6b-70bdc36a44eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.798462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4fd92438-e764-453c-ac6b-70bdc36a44eb None None] MFA Rules not processed for user `5fb568fb6097445ea9c04eb33bbd48fd`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:23.799698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4fd92438-e764-453c-ac6b-70bdc36a44eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:23.862433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1182/2345] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:23.874313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea1971-a41e-4b24-bb14-f6d52f7d73e4 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:23.876063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-37ea1971-a41e-4b24-bb14-f6d52f7d73e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:23.951634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea1971-a41e-4b24-bb14-f6d52f7d73e4 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:23.957132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa1f8e-e540-43df-ab9d-e6bff5f5aad2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:23.957809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa1f8e-e540-43df-ab9d-e6bff5f5aad2 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:23.958096 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa1f8e-e540-43df-ab9d-e6bff5f5aad2 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:23.958377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa1f8e-e540-43df-ab9d-e6bff5f5aad2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:23.966020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa1f8e-e540-43df-ab9d-e6bff5f5aad2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:23.967631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa1f8e-e540-43df-ab9d-e6bff5f5aad2 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:23.976288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1164/2346] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:24.197788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd92438-e764-453c-ac6b-70bdc36a44eb None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:24.259733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd92438-e764-453c-ac6b-70bdc36a44eb None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:24.264223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:24.265084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:24.265364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:24.265640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] PATH_INFO: `/v3/users/5fb568fb6097445ea9c04eb33bbd48fd` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:24.266970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb568fb6097445ea9c04eb33bbd48fd)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:24.268942 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:24.271261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:24.317868 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb568fb6097445ea9c04eb33bbd48fd'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.325729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb568fb6097445ea9c04eb33bbd48fd'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.332760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb568fb6097445ea9c04eb33bbd48fd'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.333395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb568fb6097445ea9c04eb33bbd48fd'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.345113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb568fb6097445ea9c04eb33bbd48fd'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.351198 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1183/2347] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:58:24 2022] DELETE /identity/v3/users/5fb568fb6097445ea9c04eb33bbd48fd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:24.356448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1820f-7172-4075-9046-b5ebd408c7c6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:24.356747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1820f-7172-4075-9046-b5ebd408c7c6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:24.357028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1820f-7172-4075-9046-b5ebd408c7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:24.376664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-55e1820f-7172-4075-9046-b5ebd408c7c6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:24.377872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-55e1820f-7172-4075-9046-b5ebd408c7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:24.435454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1165/2348] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:24.444941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:24.446331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:24.509151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075753-d74b-4897-8a29-df62e56f0eaa admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:24.521972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f9e37-84b9-4727-a8ad-8ffbec6c0bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:24.521972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f9e37-84b9-4727-a8ad-8ffbec6c0bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:24.521972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f9e37-84b9-4727-a8ad-8ffbec6c0bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:24.521972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f9e37-84b9-4727-a8ad-8ffbec6c0bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:24.530057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f9e37-84b9-4727-a8ad-8ffbec6c0bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:24.531532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f9e37-84b9-4727-a8ad-8ffbec6c0bb1 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:24.541144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1184/2349] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:24.805960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1820f-7172-4075-9046-b5ebd408c7c6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:24.807479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-55e1820f-7172-4075-9046-b5ebd408c7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:24.821461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1820f-7172-4075-9046-b5ebd408c7c6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:24.827290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:24.828165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:24.828490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:24.828902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] PATH_INFO: `/v3/projects/117dafee744049e38fa222d261381e71` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:24.830038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=117dafee744049e38fa222d261381e71)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:24.831983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:24.863701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:24.880260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '117dafee744049e38fa222d261381e71', 'request_id': 'req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.886370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '117dafee744049e38fa222d261381e71', 'request_id': 'req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.894040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '117dafee744049e38fa222d261381e71', 'request_id': 'req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.905238 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] The token cache is being invalidate because project 117dafee744049e38fa222d261381e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:58:24.905555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.906177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e83f7d7a-bfb6-494f-836b-1befd65e3e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:24.907750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1166/2350] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:58:24 2022] DELETE /identity/v3/projects/117dafee744049e38fa222d261381e71 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:24.912895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57a215-6149-4cb6-b32e-8858fa9f6e5f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:24.913190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57a215-6149-4cb6-b32e-8858fa9f6e5f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:24.913472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57a215-6149-4cb6-b32e-8858fa9f6e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:24.930746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5a57a215-6149-4cb6-b32e-8858fa9f6e5f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:24.931969 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5a57a215-6149-4cb6-b32e-8858fa9f6e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:24.990618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1185/2351] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:24.998219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81d33-b290-4a36-add5-dbcaf148c667 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:24.998510 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81d33-b290-4a36-add5-dbcaf148c667 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:24.998934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81d33-b290-4a36-add5-dbcaf148c667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.017122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c3a81d33-b290-4a36-add5-dbcaf148c667 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:25.018304 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c3a81d33-b290-4a36-add5-dbcaf148c667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:25.026012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1167/2352] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:58:25.030506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57a215-6149-4cb6-b32e-8858fa9f6e5f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.040172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57a215-6149-4cb6-b32e-8858fa9f6e5f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.044680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0faef-f6a5-4a1a-bb10-d8dd7a640b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.045320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0faef-f6a5-4a1a-bb10-d8dd7a640b69 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.045599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0faef-f6a5-4a1a-bb10-d8dd7a640b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.045875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0faef-f6a5-4a1a-bb10-d8dd7a640b69 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.046589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0faef-f6a5-4a1a-bb10-d8dd7a640b69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:58:25.046928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0faef-f6a5-4a1a-bb10-d8dd7a640b69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.048523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0faef-f6a5-4a1a-bb10-d8dd7a640b69 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.059924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1186/2353] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:58:25.063869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81d33-b290-4a36-add5-dbcaf148c667 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.072865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81d33-b290-4a36-add5-dbcaf148c667 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.078009 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec30b4-312f-4a84-b33d-c7993c4ac369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.078647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec30b4-312f-4a84-b33d-c7993c4ac369 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.078959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec30b4-312f-4a84-b33d-c7993c4ac369 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.079237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec30b4-312f-4a84-b33d-c7993c4ac369 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.079944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec30b4-312f-4a84-b33d-c7993c4ac369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.081817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec30b4-312f-4a84-b33d-c7993c4ac369 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.107268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1168/2354] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:58:25 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:58:25.111149 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0faef-f6a5-4a1a-bb10-d8dd7a640b69 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.121045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0faef-f6a5-4a1a-bb10-d8dd7a640b69 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.128728 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:25.178746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b7260-6993-4762-b374-8172b8319f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.179522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b7260-6993-4762-b374-8172b8319f2f admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.179808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b7260-6993-4762-b374-8172b8319f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.180087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b7260-6993-4762-b374-8172b8319f2f admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.180777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b7260-6993-4762-b374-8172b8319f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.182690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b7260-6993-4762-b374-8172b8319f2f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.203292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1187/2355] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:58:25 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:58:25.207425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec30b4-312f-4a84-b33d-c7993c4ac369 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.218775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec30b4-312f-4a84-b33d-c7993c4ac369 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.224618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f858f9-5c68-4b68-adf9-41506ee2cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.225315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f858f9-5c68-4b68-adf9-41506ee2cfde admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.225610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f858f9-5c68-4b68-adf9-41506ee2cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.225892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f858f9-5c68-4b68-adf9-41506ee2cfde admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.226652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f858f9-5c68-4b68-adf9-41506ee2cfde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.228286 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f858f9-5c68-4b68-adf9-41506ee2cfde admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.239346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1169/2356] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:25.243318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b7260-6993-4762-b374-8172b8319f2f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.253811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b7260-6993-4762-b374-8172b8319f2f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.258416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6105a-35ed-4228-9c8c-dfabf42ca546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.259108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6105a-35ed-4228-9c8c-dfabf42ca546 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.259393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6105a-35ed-4228-9c8c-dfabf42ca546 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.259667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6105a-35ed-4228-9c8c-dfabf42ca546 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.260391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6105a-35ed-4228-9c8c-dfabf42ca546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.262594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6105a-35ed-4228-9c8c-dfabf42ca546 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.272419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1188/2357] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:25.276779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f858f9-5c68-4b68-adf9-41506ee2cfde admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.286973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f858f9-5c68-4b68-adf9-41506ee2cfde admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.291637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c9270-a99c-4e2b-8edc-5ef9b7899cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.292511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c9270-a99c-4e2b-8edc-5ef9b7899cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.292802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c9270-a99c-4e2b-8edc-5ef9b7899cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.293109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c9270-a99c-4e2b-8edc-5ef9b7899cc9 admin admin] PATH_INFO: `/v3/projects/148eb054d6b44042b9cd603f118c94a7/users/f6402940aa4443829fb27ef28084e1d5/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.307460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c9270-a99c-4e2b-8edc-5ef9b7899cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=148eb054d6b44042b9cd603f118c94a7, user_id=f6402940aa4443829fb27ef28084e1d5, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.310787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c9270-a99c-4e2b-8edc-5ef9b7899cc9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.324130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1170/2358] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:58:25 2022] PUT /identity/v3/projects/148eb054d6b44042b9cd603f118c94a7/users/f6402940aa4443829fb27ef28084e1d5/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:58:25.334789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b1afd-c76c-4190-93e2-aa59ec8a254e None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.335108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b1afd-c76c-4190-93e2-aa59ec8a254e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.335387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b1afd-c76c-4190-93e2-aa59ec8a254e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.361093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-205b1afd-c76c-4190-93e2-aa59ec8a254e None None] MFA Rules not processed for user `f6402940aa4443829fb27ef28084e1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:25.362293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-205b1afd-c76c-4190-93e2-aa59ec8a254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:25.426028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1189/2359] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:25.430462 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c9270-a99c-4e2b-8edc-5ef9b7899cc9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.492475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c9270-a99c-4e2b-8edc-5ef9b7899cc9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.497653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c8bea-0ca3-4c7a-8cc4-bb59bde86487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.498306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c8bea-0ca3-4c7a-8cc4-bb59bde86487 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.498590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c8bea-0ca3-4c7a-8cc4-bb59bde86487 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.498900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c8bea-0ca3-4c7a-8cc4-bb59bde86487 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.499594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c8bea-0ca3-4c7a-8cc4-bb59bde86487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.501504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c8bea-0ca3-4c7a-8cc4-bb59bde86487 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.527497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1171/2360] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:58:25 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:58:25.531625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b1afd-c76c-4190-93e2-aa59ec8a254e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.542964 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b1afd-c76c-4190-93e2-aa59ec8a254e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.598552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8016c2c-7377-4648-a57c-94732d416a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.599169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8016c2c-7377-4648-a57c-94732d416a92 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.599445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8016c2c-7377-4648-a57c-94732d416a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.599716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8016c2c-7377-4648-a57c-94732d416a92 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.600399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8016c2c-7377-4648-a57c-94732d416a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.602291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8016c2c-7377-4648-a57c-94732d416a92 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.622339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1190/2361] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:58:25 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:58:25.626230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c8bea-0ca3-4c7a-8cc4-bb59bde86487 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.636336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c8bea-0ca3-4c7a-8cc4-bb59bde86487 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.640879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19ecee-bc0a-41d0-a40d-e1b1694bfa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.641528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19ecee-bc0a-41d0-a40d-e1b1694bfa7f admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.641806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19ecee-bc0a-41d0-a40d-e1b1694bfa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.642098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19ecee-bc0a-41d0-a40d-e1b1694bfa7f admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.642832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19ecee-bc0a-41d0-a40d-e1b1694bfa7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.644406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19ecee-bc0a-41d0-a40d-e1b1694bfa7f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.654599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1172/2362] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:25.658409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8016c2c-7377-4648-a57c-94732d416a92 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.669028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8016c2c-7377-4648-a57c-94732d416a92 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.673678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a0a58-eb2e-48f2-b45c-a20316d5800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.674544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a0a58-eb2e-48f2-b45c-a20316d5800c admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.674856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a0a58-eb2e-48f2-b45c-a20316d5800c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.675135 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a0a58-eb2e-48f2-b45c-a20316d5800c admin admin] PATH_INFO: `/v3/projects/a07871f23b2447edaea10077d943e8f3/users/634fcd2c63014c608807cf1620bdc5fe/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.678152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:25.691593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a0a58-eb2e-48f2-b45c-a20316d5800c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07871f23b2447edaea10077d943e8f3, user_id=634fcd2c63014c608807cf1620bdc5fe, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.694272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a0a58-eb2e-48f2-b45c-a20316d5800c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.707287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1191/2363] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:58:25 2022] PUT /identity/v3/projects/a07871f23b2447edaea10077d943e8f3/users/634fcd2c63014c608807cf1620bdc5fe/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:25.711270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19ecee-bc0a-41d0-a40d-e1b1694bfa7f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.721221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19ecee-bc0a-41d0-a40d-e1b1694bfa7f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.745052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:25.779522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31d3bc-482c-4213-a224-b895d080acfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.780191 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d3bc-482c-4213-a224-b895d080acfc admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.780471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d3bc-482c-4213-a224-b895d080acfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.780749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d3bc-482c-4213-a224-b895d080acfc admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.781465 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31d3bc-482c-4213-a224-b895d080acfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.783073 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31d3bc-482c-4213-a224-b895d080acfc admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.793746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1173/2364] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:25.798008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a0a58-eb2e-48f2-b45c-a20316d5800c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.808252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a0a58-eb2e-48f2-b45c-a20316d5800c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:25.813530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2e69-872c-4eb1-a7a5-32aa5f77d0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:25.814405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2e69-872c-4eb1-a7a5-32aa5f77d0ef admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.814688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2e69-872c-4eb1-a7a5-32aa5f77d0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.815003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2e69-872c-4eb1-a7a5-32aa5f77d0ef admin admin] PATH_INFO: `/v3/projects/a07871f23b2447edaea10077d943e8f3/users/634fcd2c63014c608807cf1620bdc5fe/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.816980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c2e69-872c-4eb1-a7a5-32aa5f77d0ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07871f23b2447edaea10077d943e8f3, user_id=634fcd2c63014c608807cf1620bdc5fe, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:25.819650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c2e69-872c-4eb1-a7a5-32aa5f77d0ef admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:25.829667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1192/2365] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:58:25 2022] PUT /identity/v3/projects/a07871f23b2447edaea10077d943e8f3/users/634fcd2c63014c608807cf1620bdc5fe/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:25.840849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb55a0-a8a8-482e-92f0-ee87f25d5263 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:25.841136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb55a0-a8a8-482e-92f0-ee87f25d5263 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:25.841411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb55a0-a8a8-482e-92f0-ee87f25d5263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:25.868045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-77fb55a0-a8a8-482e-92f0-ee87f25d5263 None None] MFA Rules not processed for user `634fcd2c63014c608807cf1620bdc5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:25.869137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-77fb55a0-a8a8-482e-92f0-ee87f25d5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:25.935747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1174/2366] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:25.947062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2e69-872c-4eb1-a7a5-32aa5f77d0ef admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:25.949126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9e4c2e69-872c-4eb1-a7a5-32aa5f77d0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:26.026594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2e69-872c-4eb1-a7a5-32aa5f77d0ef admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:26.030906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaecf2-f80d-4ab0-9cbd-21bd089630f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:26.031544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaecf2-f80d-4ab0-9cbd-21bd089630f1 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:26.031826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaecf2-f80d-4ab0-9cbd-21bd089630f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:26.032107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaecf2-f80d-4ab0-9cbd-21bd089630f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:26.041212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebaecf2-f80d-4ab0-9cbd-21bd089630f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:26.042635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebaecf2-f80d-4ab0-9cbd-21bd089630f1 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:26.052509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1193/2367] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:26.120261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb55a0-a8a8-482e-92f0-ee87f25d5263 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:26.129776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb55a0-a8a8-482e-92f0-ee87f25d5263 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:26.133390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7a409-d9e7-4928-a6f4-3f24c9f80376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:26.134023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7a409-d9e7-4928-a6f4-3f24c9f80376 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:26.134308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7a409-d9e7-4928-a6f4-3f24c9f80376 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:26.134589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7a409-d9e7-4928-a6f4-3f24c9f80376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:26.191977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7a409-d9e7-4928-a6f4-3f24c9f80376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:26.193452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7a409-d9e7-4928-a6f4-3f24c9f80376 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:26.202441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1175/2368] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:36.048360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:36.198381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=79 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:36.993618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaecf2-f80d-4ab0-9cbd-21bd089630f1 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:37.054945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaecf2-f80d-4ab0-9cbd-21bd089630f1 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:37.059541 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:37.060428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:37.060712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:37.060990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] PATH_INFO: `/v3/users/f6402940aa4443829fb27ef28084e1d5` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:37.062169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6402940aa4443829fb27ef28084e1d5)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:37.064103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:37.109372 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6402940aa4443829fb27ef28084e1d5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.116199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6402940aa4443829fb27ef28084e1d5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.123026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6402940aa4443829fb27ef28084e1d5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.123657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6402940aa4443829fb27ef28084e1d5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.135285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6402940aa4443829fb27ef28084e1d5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.141806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1194/2369] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:58:36 2022] DELETE /identity/v3/users/f6402940aa4443829fb27ef28084e1d5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:37.146349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade951ba-6090-4369-9992-59aa4f896b94 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:37.146646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade951ba-6090-4369-9992-59aa4f896b94 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:37.146975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade951ba-6090-4369-9992-59aa4f896b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:37.165786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ade951ba-6090-4369-9992-59aa4f896b94 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:37.167015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ade951ba-6090-4369-9992-59aa4f896b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:37.225031 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1176/2370] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:37.234494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:37.235902 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:37.298122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa3c6f-d388-4b70-8e21-6df46ea2d638 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:37.301819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179bc76-19c8-4c93-84bf-acae0b7f2ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:37.302465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179bc76-19c8-4c93-84bf-acae0b7f2ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:37.302769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179bc76-19c8-4c93-84bf-acae0b7f2ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:37.303057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179bc76-19c8-4c93-84bf-acae0b7f2ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:37.312518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179bc76-19c8-4c93-84bf-acae0b7f2ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:37.313954 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179bc76-19c8-4c93-84bf-acae0b7f2ab4 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:37.323929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1195/2371] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:37.804700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade951ba-6090-4369-9992-59aa4f896b94 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:37.806249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ade951ba-6090-4369-9992-59aa4f896b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:37.819501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade951ba-6090-4369-9992-59aa4f896b94 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:37.824674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:37.825543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:37.825848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:37.826150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] PATH_INFO: `/v3/projects/148eb054d6b44042b9cd603f118c94a7` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:37.827388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] RBAC: Authorizing `identity:delete_project(project_id=148eb054d6b44042b9cd603f118c94a7)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:37.829283 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:37.876023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '148eb054d6b44042b9cd603f118c94a7', 'request_id': 'req-06be3383-bee0-4e59-920a-409d1681540f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.882043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '148eb054d6b44042b9cd603f118c94a7', 'request_id': 'req-06be3383-bee0-4e59-920a-409d1681540f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.888565 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '148eb054d6b44042b9cd603f118c94a7', 'request_id': 'req-06be3383-bee0-4e59-920a-409d1681540f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.898703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] The token cache is being invalidate because project 148eb054d6b44042b9cd603f118c94a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:58:37.899020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.899639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:37.901159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1177/2372] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:58:37 2022] DELETE /identity/v3/projects/148eb054d6b44042b9cd603f118c94a7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:37.905752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179bc76-19c8-4c93-84bf-acae0b7f2ab4 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:37.915546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179bc76-19c8-4c93-84bf-acae0b7f2ab4 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:37.935363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:37.975584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:37.976446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:37.976720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:37.976991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] PATH_INFO: `/v3/users/634fcd2c63014c608807cf1620bdc5fe` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:37.978158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=634fcd2c63014c608807cf1620bdc5fe)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:37.980094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:38.027147 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '634fcd2c63014c608807cf1620bdc5fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.035031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '634fcd2c63014c608807cf1620bdc5fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.041918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '634fcd2c63014c608807cf1620bdc5fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.042543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '634fcd2c63014c608807cf1620bdc5fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.054713 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-611cb931-f868-4268-9d48-a8bc6b6df3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '634fcd2c63014c608807cf1620bdc5fe'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.060552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1196/2373] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:58:37 2022] DELETE /identity/v3/users/634fcd2c63014c608807cf1620bdc5fe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:38.327085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:38.328556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:38.344150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be3383-bee0-4e59-920a-409d1681540f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:38.400779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:38.401676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.401959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.402233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] PATH_INFO: `/v3/projects/a07871f23b2447edaea10077d943e8f3` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.404064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07871f23b2447edaea10077d943e8f3)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:38.405959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:38.440590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:38.458913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07871f23b2447edaea10077d943e8f3', 'request_id': 'req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.465005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07871f23b2447edaea10077d943e8f3', 'request_id': 'req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.472893 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07871f23b2447edaea10077d943e8f3', 'request_id': 'req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.483535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] The token cache is being invalidate because project a07871f23b2447edaea10077d943e8f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:58:38.483822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.484435 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-9d31f8f1-ba30-49ae-948d-d5b5ff41a5e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:38.485958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1178/2374] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:58:38 2022] DELETE /identity/v3/projects/a07871f23b2447edaea10077d943e8f3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:38.490906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a96b6-5b87-4f1c-b240-b3e630d1accf None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.491190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a96b6-5b87-4f1c-b240-b3e630d1accf None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.491474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a96b6-5b87-4f1c-b240-b3e630d1accf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.508668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6c4a96b6-5b87-4f1c-b240-b3e630d1accf None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:38.509870 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6c4a96b6-5b87-4f1c-b240-b3e630d1accf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:38.567251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1197/2375] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:38.575274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efa98f-bdac-4883-941f-b8fa4524bb40 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.575571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efa98f-bdac-4883-941f-b8fa4524bb40 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.575857 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efa98f-bdac-4883-941f-b8fa4524bb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.594490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d6efa98f-bdac-4883-941f-b8fa4524bb40 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:38.595731 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d6efa98f-bdac-4883-941f-b8fa4524bb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:38.603611 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1179/2376] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:38.607679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a96b6-5b87-4f1c-b240-b3e630d1accf None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:38.617263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a96b6-5b87-4f1c-b240-b3e630d1accf None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:38.621883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aae6c2-42e1-42e9-811f-1ffe70483546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:38.622530 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aae6c2-42e1-42e9-811f-1ffe70483546 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.622841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aae6c2-42e1-42e9-811f-1ffe70483546 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.623123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aae6c2-42e1-42e9-811f-1ffe70483546 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.623841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aae6c2-42e1-42e9-811f-1ffe70483546 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:58:38.624151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aae6c2-42e1-42e9-811f-1ffe70483546 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:38.625737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aae6c2-42e1-42e9-811f-1ffe70483546 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:38.637604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1198/2377] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:58:38.641809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efa98f-bdac-4883-941f-b8fa4524bb40 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:38.650965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efa98f-bdac-4883-941f-b8fa4524bb40 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:38.656323 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845c289-2b8f-4292-a4d3-3ec0c09ff031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:38.657006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845c289-2b8f-4292-a4d3-3ec0c09ff031 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.657292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845c289-2b8f-4292-a4d3-3ec0c09ff031 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.657572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845c289-2b8f-4292-a4d3-3ec0c09ff031 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.658260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845c289-2b8f-4292-a4d3-3ec0c09ff031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:38.660475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845c289-2b8f-4292-a4d3-3ec0c09ff031 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:38.685887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1180/2378] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:58:38 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:58:38.689922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aae6c2-42e1-42e9-811f-1ffe70483546 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:38.700347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aae6c2-42e1-42e9-811f-1ffe70483546 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:38.725394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:38.757211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dc4ab-2151-4b1f-bab5-fb6c17fd8960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:38.757874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dc4ab-2151-4b1f-bab5-fb6c17fd8960 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.758162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dc4ab-2151-4b1f-bab5-fb6c17fd8960 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.758443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dc4ab-2151-4b1f-bab5-fb6c17fd8960 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.759175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33dc4ab-2151-4b1f-bab5-fb6c17fd8960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:38.761092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33dc4ab-2151-4b1f-bab5-fb6c17fd8960 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:38.780478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1199/2379] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:58:38 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:58:38.784757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845c289-2b8f-4292-a4d3-3ec0c09ff031 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:38.795106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845c289-2b8f-4292-a4d3-3ec0c09ff031 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:38.799713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93761c00-c1ed-411f-90e8-7079be65dd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:38.800372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93761c00-c1ed-411f-90e8-7079be65dd1e admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.800656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93761c00-c1ed-411f-90e8-7079be65dd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.800936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93761c00-c1ed-411f-90e8-7079be65dd1e admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.801666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93761c00-c1ed-411f-90e8-7079be65dd1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:38.803301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93761c00-c1ed-411f-90e8-7079be65dd1e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:38.814180 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1181/2380] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:38.817852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dc4ab-2151-4b1f-bab5-fb6c17fd8960 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:38.828292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dc4ab-2151-4b1f-bab5-fb6c17fd8960 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:38.832926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8ca73-492d-4812-b045-c851fd757ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:38.833571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ca73-492d-4812-b045-c851fd757ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.833860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ca73-492d-4812-b045-c851fd757ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.834141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ca73-492d-4812-b045-c851fd757ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.834895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8ca73-492d-4812-b045-c851fd757ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:38.836487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8ca73-492d-4812-b045-c851fd757ec3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:38.846217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1200/2381] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:38.850014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93761c00-c1ed-411f-90e8-7079be65dd1e admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:38.860193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93761c00-c1ed-411f-90e8-7079be65dd1e admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:38.864879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e76990-a4bb-4e77-aa5e-cb95e2d535d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:38.865758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e76990-a4bb-4e77-aa5e-cb95e2d535d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.866043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e76990-a4bb-4e77-aa5e-cb95e2d535d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.866324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e76990-a4bb-4e77-aa5e-cb95e2d535d4 admin admin] PATH_INFO: `/v3/projects/3da8f788e6e84356819d876e34dc3695/users/af3fb6e4d61c40c0a76418b4ab2bbf50/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.881281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e76990-a4bb-4e77-aa5e-cb95e2d535d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3da8f788e6e84356819d876e34dc3695, user_id=af3fb6e4d61c40c0a76418b4ab2bbf50, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:38.883950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e76990-a4bb-4e77-aa5e-cb95e2d535d4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:38.898243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1182/2382] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:58:38 2022] PUT /identity/v3/projects/3da8f788e6e84356819d876e34dc3695/users/af3fb6e4d61c40c0a76418b4ab2bbf50/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:38.909721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fb062-fc4f-4dcd-b53f-cac96b97f571 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:38.910014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fb062-fc4f-4dcd-b53f-cac96b97f571 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:38.910295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fb062-fc4f-4dcd-b53f-cac96b97f571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:38.936664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e95fb062-fc4f-4dcd-b53f-cac96b97f571 None None] MFA Rules not processed for user `af3fb6e4d61c40c0a76418b4ab2bbf50`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:38.937899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e95fb062-fc4f-4dcd-b53f-cac96b97f571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:39.004877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1201/2383] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:39.009429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e76990-a4bb-4e77-aa5e-cb95e2d535d4 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:39.070474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e76990-a4bb-4e77-aa5e-cb95e2d535d4 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:39.075094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c23f5a-7f61-4079-8959-42bfb0bb54f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:39.075772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c23f5a-7f61-4079-8959-42bfb0bb54f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:39.076056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c23f5a-7f61-4079-8959-42bfb0bb54f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:39.076332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c23f5a-7f61-4079-8959-42bfb0bb54f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:39.077012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c23f5a-7f61-4079-8959-42bfb0bb54f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:39.078984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c23f5a-7f61-4079-8959-42bfb0bb54f2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:39.104481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1183/2384] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:58:39 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:58:39.109061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fb062-fc4f-4dcd-b53f-cac96b97f571 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:39.120240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fb062-fc4f-4dcd-b53f-cac96b97f571 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:39.175625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f82ab-a70f-4f6d-a9af-cec026215747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:39.176293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f82ab-a70f-4f6d-a9af-cec026215747 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:39.176580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f82ab-a70f-4f6d-a9af-cec026215747 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:39.176888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f82ab-a70f-4f6d-a9af-cec026215747 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:39.177585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f82ab-a70f-4f6d-a9af-cec026215747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:39.179550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f82ab-a70f-4f6d-a9af-cec026215747 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:39.201933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1202/2385] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:58:39 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:58:39.205683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c23f5a-7f61-4079-8959-42bfb0bb54f2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:39.216423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c23f5a-7f61-4079-8959-42bfb0bb54f2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:39.221008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74649cea-193b-4877-b025-bd0ef052adc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:39.221667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74649cea-193b-4877-b025-bd0ef052adc9 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:39.221952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74649cea-193b-4877-b025-bd0ef052adc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:39.222234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74649cea-193b-4877-b025-bd0ef052adc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:39.222995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74649cea-193b-4877-b025-bd0ef052adc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:39.224585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74649cea-193b-4877-b025-bd0ef052adc9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:39.235458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1184/2386] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:39.239584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f82ab-a70f-4f6d-a9af-cec026215747 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:39.250064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f82ab-a70f-4f6d-a9af-cec026215747 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:39.254853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25149c3-ebe1-4c10-978b-ece411a772ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:39.255722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25149c3-ebe1-4c10-978b-ece411a772ce admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:39.256003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25149c3-ebe1-4c10-978b-ece411a772ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:39.256280 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25149c3-ebe1-4c10-978b-ece411a772ce admin admin] PATH_INFO: `/v3/projects/27ed6cfe731d4677befa2c6c1bd8e933/users/9bc2af4ffb794b8c88a88f0073f70cb3/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:39.270551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25149c3-ebe1-4c10-978b-ece411a772ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ed6cfe731d4677befa2c6c1bd8e933, user_id=9bc2af4ffb794b8c88a88f0073f70cb3, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:39.273239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25149c3-ebe1-4c10-978b-ece411a772ce admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:39.286412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1203/2387] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:58:39 2022] PUT /identity/v3/projects/27ed6cfe731d4677befa2c6c1bd8e933/users/9bc2af4ffb794b8c88a88f0073f70cb3/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:58:39.290421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74649cea-193b-4877-b025-bd0ef052adc9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:39.305751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74649cea-193b-4877-b025-bd0ef052adc9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:39.326960 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:39.363449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834d67f-fd5f-421c-a7b8-650840c5482d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:39.364111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d67f-fd5f-421c-a7b8-650840c5482d admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:39.364394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d67f-fd5f-421c-a7b8-650840c5482d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:39.364673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d67f-fd5f-421c-a7b8-650840c5482d admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:39.365393 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834d67f-fd5f-421c-a7b8-650840c5482d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:39.367004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834d67f-fd5f-421c-a7b8-650840c5482d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:39.377252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1185/2388] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:39.381515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25149c3-ebe1-4c10-978b-ece411a772ce admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:39.387416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:39.393494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25149c3-ebe1-4c10-978b-ece411a772ce admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:39.398673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74d051-1d8d-47d3-9a92-391cb11a7a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:39.399607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74d051-1d8d-47d3-9a92-391cb11a7a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:39.399890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74d051-1d8d-47d3-9a92-391cb11a7a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:39.400169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74d051-1d8d-47d3-9a92-391cb11a7a5e admin admin] PATH_INFO: `/v3/projects/27ed6cfe731d4677befa2c6c1bd8e933/users/9bc2af4ffb794b8c88a88f0073f70cb3/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:39.402130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74d051-1d8d-47d3-9a92-391cb11a7a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ed6cfe731d4677befa2c6c1bd8e933, user_id=9bc2af4ffb794b8c88a88f0073f70cb3, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:39.405396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74d051-1d8d-47d3-9a92-391cb11a7a5e admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:39.415966 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1204/2389] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:58:39 2022] PUT /identity/v3/projects/27ed6cfe731d4677befa2c6c1bd8e933/users/9bc2af4ffb794b8c88a88f0073f70cb3/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:39.426598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f13c5-6242-410f-ba67-71d4d8fa7edb None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:39.426909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f13c5-6242-410f-ba67-71d4d8fa7edb None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:39.427194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f13c5-6242-410f-ba67-71d4d8fa7edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:39.454772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-674f13c5-6242-410f-ba67-71d4d8fa7edb None None] MFA Rules not processed for user `9bc2af4ffb794b8c88a88f0073f70cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:39.456000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-674f13c5-6242-410f-ba67-71d4d8fa7edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:39.523098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1186/2390] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:39.534393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74d051-1d8d-47d3-9a92-391cb11a7a5e admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:39.536108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cd74d051-1d8d-47d3-9a92-391cb11a7a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:39.603614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74d051-1d8d-47d3-9a92-391cb11a7a5e admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:39.607876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d4a6c-4a26-4aa1-b3d3-3e88d32ebecf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:39.608553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d4a6c-4a26-4aa1-b3d3-3e88d32ebecf service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:39.608834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d4a6c-4a26-4aa1-b3d3-3e88d32ebecf service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:39.609111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d4a6c-4a26-4aa1-b3d3-3e88d32ebecf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:39.663703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d4a6c-4a26-4aa1-b3d3-3e88d32ebecf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:39.665141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d4a6c-4a26-4aa1-b3d3-3e88d32ebecf service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:39.674500 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1205/2391] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:44.867637 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f13c5-6242-410f-ba67-71d4d8fa7edb None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:44.877663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f13c5-6242-410f-ba67-71d4d8fa7edb None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:44.881301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52d053-0a73-4e53-bad2-925656b30cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:44.881935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52d053-0a73-4e53-bad2-925656b30cbf service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:44.882211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52d053-0a73-4e53-bad2-925656b30cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:44.882482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52d053-0a73-4e53-bad2-925656b30cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:44.892173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52d053-0a73-4e53-bad2-925656b30cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:44.893626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52d053-0a73-4e53-bad2-925656b30cbf service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:44.904341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1187/2392] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:49.669923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:49.745709 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d4a6c-4a26-4aa1-b3d3-3e88d32ebecf service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:49.807532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d4a6c-4a26-4aa1-b3d3-3e88d32ebecf service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:49.811960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:49.812814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:49.813091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:49.813362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] PATH_INFO: `/v3/users/af3fb6e4d61c40c0a76418b4ab2bbf50` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:49.814516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3fb6e4d61c40c0a76418b4ab2bbf50)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:49.816444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:49.861624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3fb6e4d61c40c0a76418b4ab2bbf50'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:49.868929 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3fb6e4d61c40c0a76418b4ab2bbf50'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:49.876491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3fb6e4d61c40c0a76418b4ab2bbf50'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:49.877127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3fb6e4d61c40c0a76418b4ab2bbf50'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:49.889014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3fb6e4d61c40c0a76418b4ab2bbf50'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:49.894984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1206/2393] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:58:49 2022] DELETE /identity/v3/users/af3fb6e4d61c40c0a76418b4ab2bbf50 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:49.899436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b7f3f-f38d-40b8-b519-67a2efb7e949 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:49.899903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b7f3f-f38d-40b8-b519-67a2efb7e949 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:49.900006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b7f3f-f38d-40b8-b519-67a2efb7e949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:49.917809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e41b7f3f-f38d-40b8-b519-67a2efb7e949 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:49.919642 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e41b7f3f-f38d-40b8-b519-67a2efb7e949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:49.977359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1188/2394] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:58:49.987150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:49.988560 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:50.050846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbded8-2487-4af8-ad61-a308549c1b21 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:50.054444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b2011-2bb2-42d8-9f2d-a201a53b3268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:50.055115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b2011-2bb2-42d8-9f2d-a201a53b3268 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:50.055395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b2011-2bb2-42d8-9f2d-a201a53b3268 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:50.055668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b2011-2bb2-42d8-9f2d-a201a53b3268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:50.064936 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b2011-2bb2-42d8-9f2d-a201a53b3268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:50.066352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b2011-2bb2-42d8-9f2d-a201a53b3268 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:50.076233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1207/2395] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:50.336795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b7f3f-f38d-40b8-b519-67a2efb7e949 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:50.338248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e41b7f3f-f38d-40b8-b519-67a2efb7e949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:50.349980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b7f3f-f38d-40b8-b519-67a2efb7e949 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:50.354624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:50.355521 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:50.355806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:50.356610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] PATH_INFO: `/v3/projects/3da8f788e6e84356819d876e34dc3695` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:50.357806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3da8f788e6e84356819d876e34dc3695)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:50.359767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:50.393359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:50.409754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3da8f788e6e84356819d876e34dc3695', 'request_id': 'req-ec36beb8-08de-4969-a709-286a0df4257f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.416113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3da8f788e6e84356819d876e34dc3695', 'request_id': 'req-ec36beb8-08de-4969-a709-286a0df4257f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.423033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3da8f788e6e84356819d876e34dc3695', 'request_id': 'req-ec36beb8-08de-4969-a709-286a0df4257f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.433240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] The token cache is being invalidate because project 3da8f788e6e84356819d876e34dc3695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:58:50.433530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.434170 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.435759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1189/2396] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:58:50 2022] DELETE /identity/v3/projects/3da8f788e6e84356819d876e34dc3695 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:50.439865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b2011-2bb2-42d8-9f2d-a201a53b3268 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:50.450630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b2011-2bb2-42d8-9f2d-a201a53b3268 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:50.468867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:50.508337 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:50.509128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:50.509409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:50.509687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] PATH_INFO: `/v3/users/9bc2af4ffb794b8c88a88f0073f70cb3` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:50.510927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc2af4ffb794b8c88a88f0073f70cb3)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:50.513400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:50.557734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc2af4ffb794b8c88a88f0073f70cb3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.564808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc2af4ffb794b8c88a88f0073f70cb3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.572091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc2af4ffb794b8c88a88f0073f70cb3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.572703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc2af4ffb794b8c88a88f0073f70cb3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.585009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-17d29698-9a5c-4737-ba9a-cb04f4f83959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc2af4ffb794b8c88a88f0073f70cb3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.591616 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1208/2397] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 15:58:50 2022] DELETE /identity/v3/users/9bc2af4ffb794b8c88a88f0073f70cb3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:50.857228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:50.858752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:50.876634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36beb8-08de-4969-a709-286a0df4257f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:50.934694 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:50.935591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:50.935878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:50.936156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] PATH_INFO: `/v3/projects/27ed6cfe731d4677befa2c6c1bd8e933` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:50.937317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=27ed6cfe731d4677befa2c6c1bd8e933)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:50.939277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:50.983587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27ed6cfe731d4677befa2c6c1bd8e933', 'request_id': 'req-88083a79-8e15-4981-84d6-71962303f0eb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.990436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27ed6cfe731d4677befa2c6c1bd8e933', 'request_id': 'req-88083a79-8e15-4981-84d6-71962303f0eb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:50.997772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27ed6cfe731d4677befa2c6c1bd8e933', 'request_id': 'req-88083a79-8e15-4981-84d6-71962303f0eb'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:51.008357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] The token cache is being invalidate because project 27ed6cfe731d4677befa2c6c1bd8e933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:58:51.008645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:51.009265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-88083a79-8e15-4981-84d6-71962303f0eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:58:51.010822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1190/2398] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 15:58:50 2022] DELETE /identity/v3/projects/27ed6cfe731d4677befa2c6c1bd8e933 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:51.017267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3965171-b1c0-45bc-a5bb-031431187cae None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.017754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3965171-b1c0-45bc-a5bb-031431187cae None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.017850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3965171-b1c0-45bc-a5bb-031431187cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.035723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b3965171-b1c0-45bc-a5bb-031431187cae None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:51.036921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b3965171-b1c0-45bc-a5bb-031431187cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:51.097412 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1209/2399] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:51.105572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053f86b-8474-4685-b517-bbc8409e7e98 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.106394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053f86b-8474-4685-b517-bbc8409e7e98 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.106677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053f86b-8474-4685-b517-bbc8409e7e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.125078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d053f86b-8474-4685-b517-bbc8409e7e98 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:51.126266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d053f86b-8474-4685-b517-bbc8409e7e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:51.134106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1191/2400] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:51.138249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3965171-b1c0-45bc-a5bb-031431187cae None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:51.147931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3965171-b1c0-45bc-a5bb-031431187cae None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:51.152509 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310df5c2-763c-4b99-82d1-580fb873e762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:51.153190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310df5c2-763c-4b99-82d1-580fb873e762 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.153475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310df5c2-763c-4b99-82d1-580fb873e762 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.153754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310df5c2-763c-4b99-82d1-580fb873e762 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.154472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310df5c2-763c-4b99-82d1-580fb873e762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:58:51.154812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310df5c2-763c-4b99-82d1-580fb873e762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:51.156413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310df5c2-763c-4b99-82d1-580fb873e762 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:51.168013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1210/2401] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 15:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:58:51.173038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053f86b-8474-4685-b517-bbc8409e7e98 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:51.182240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053f86b-8474-4685-b517-bbc8409e7e98 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:51.187549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05615c-7001-47b7-9b25-d08012c75ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:51.188197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05615c-7001-47b7-9b25-d08012c75ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.188481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05615c-7001-47b7-9b25-d08012c75ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.188760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05615c-7001-47b7-9b25-d08012c75ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.189462 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05615c-7001-47b7-9b25-d08012c75ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:51.191398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05615c-7001-47b7-9b25-d08012c75ea2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:51.217626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1192/2402] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 15:58:51 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:58:51.220657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310df5c2-763c-4b99-82d1-580fb873e762 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:51.230517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310df5c2-763c-4b99-82d1-580fb873e762 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:51.257189 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:51.289387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427271b-b15c-450f-9dad-e29354d7b0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:51.289991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427271b-b15c-450f-9dad-e29354d7b0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.290273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427271b-b15c-450f-9dad-e29354d7b0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.290551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427271b-b15c-450f-9dad-e29354d7b0c7 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.291274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427271b-b15c-450f-9dad-e29354d7b0c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:51.293808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427271b-b15c-450f-9dad-e29354d7b0c7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:51.313299 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1211/2403] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 15:58:51 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:58:51.317559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05615c-7001-47b7-9b25-d08012c75ea2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:51.328271 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05615c-7001-47b7-9b25-d08012c75ea2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:51.332769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd983ef9-c0ac-4930-b035-225448cb350b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:51.333415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd983ef9-c0ac-4930-b035-225448cb350b admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.333694 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd983ef9-c0ac-4930-b035-225448cb350b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.333969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd983ef9-c0ac-4930-b035-225448cb350b admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.334693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd983ef9-c0ac-4930-b035-225448cb350b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:51.336291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd983ef9-c0ac-4930-b035-225448cb350b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:51.346459 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1193/2404] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:51.350294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427271b-b15c-450f-9dad-e29354d7b0c7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:51.363797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427271b-b15c-450f-9dad-e29354d7b0c7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:51.369373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6f39f-f426-4cfd-b0d3-94d19fc2f7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:51.370042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6f39f-f426-4cfd-b0d3-94d19fc2f7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.370323 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6f39f-f426-4cfd-b0d3-94d19fc2f7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.370601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6f39f-f426-4cfd-b0d3-94d19fc2f7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.373652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6f39f-f426-4cfd-b0d3-94d19fc2f7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:51.373652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6f39f-f426-4cfd-b0d3-94d19fc2f7d0 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:51.384190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1212/2405] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 15:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:58:51.388389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd983ef9-c0ac-4930-b035-225448cb350b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:51.398309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd983ef9-c0ac-4930-b035-225448cb350b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:51.403608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2fdda-f369-4ab3-a1ab-bfcf740d7419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:51.404481 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2fdda-f369-4ab3-a1ab-bfcf740d7419 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.404768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2fdda-f369-4ab3-a1ab-bfcf740d7419 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.405049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2fdda-f369-4ab3-a1ab-bfcf740d7419 admin admin] PATH_INFO: `/v3/projects/4babb3ea9898402ba7898395d59b4beb/users/00609e23ec8d4eadb9709b4e497810e3/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.407319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:58:51.422620 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2fdda-f369-4ab3-a1ab-bfcf740d7419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4babb3ea9898402ba7898395d59b4beb, user_id=00609e23ec8d4eadb9709b4e497810e3, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:51.425326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2fdda-f369-4ab3-a1ab-bfcf740d7419 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:51.437864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1194/2406] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 15:58:51 2022] PUT /identity/v3/projects/4babb3ea9898402ba7898395d59b4beb/users/00609e23ec8d4eadb9709b4e497810e3/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:58:51.443069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3596-ef15-4c4f-8959-ee6dd4273ff8 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.443361 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3596-ef15-4c4f-8959-ee6dd4273ff8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.443640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3596-ef15-4c4f-8959-ee6dd4273ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.461308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-fcfc3596-ef15-4c4f-8959-ee6dd4273ff8 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:51.462550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fcfc3596-ef15-4c4f-8959-ee6dd4273ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:51.520566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1213/2407] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:51.530411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2fdda-f369-4ab3-a1ab-bfcf740d7419 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:51.532129 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a4a2fdda-f369-4ab3-a1ab-bfcf740d7419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:51.588609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2fdda-f369-4ab3-a1ab-bfcf740d7419 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:51.592719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b41eb6-e7df-4544-8dcb-ffb602438a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:51.593357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b41eb6-e7df-4544-8dcb-ffb602438a9b service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:51.593640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b41eb6-e7df-4544-8dcb-ffb602438a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:51.593916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b41eb6-e7df-4544-8dcb-ffb602438a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:51.602843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b41eb6-e7df-4544-8dcb-ffb602438a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:51.604263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b41eb6-e7df-4544-8dcb-ffb602438a9b service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:51.614106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1195/2408] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:52.190178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc9465-6b26-407f-ae30-c63be5bb096f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:52.190764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc9465-6b26-407f-ae30-c63be5bb096f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:52.190941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc9465-6b26-407f-ae30-c63be5bb096f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:52.217739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-55cc9465-6b26-407f-ae30-c63be5bb096f None None] MFA Rules not processed for user `00609e23ec8d4eadb9709b4e497810e3`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:52.219078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-55cc9465-6b26-407f-ae30-c63be5bb096f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:52.285004 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1214/2409] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:52.296351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b41eb6-e7df-4544-8dcb-ffb602438a9b service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:52.297891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-77b41eb6-e7df-4544-8dcb-ffb602438a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:52.356068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b41eb6-e7df-4544-8dcb-ffb602438a9b service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:52.359632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f408a28-cd2b-41d0-9397-c034f11c972f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:52.360274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408a28-cd2b-41d0-9397-c034f11c972f service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:52.360557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408a28-cd2b-41d0-9397-c034f11c972f service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:52.360843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408a28-cd2b-41d0-9397-c034f11c972f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:52.369380 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f408a28-cd2b-41d0-9397-c034f11c972f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:52.370814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f408a28-cd2b-41d0-9397-c034f11c972f service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:52.380452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1196/2410] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:58:54.761014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4cb1f-cca9-43d2-a995-7c89a3cdd31a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:54.761493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4cb1f-cca9-43d2-a995-7c89a3cdd31a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:54.761585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4cb1f-cca9-43d2-a995-7c89a3cdd31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:54.779844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b5c4cb1f-cca9-43d2-a995-7c89a3cdd31a None None] MFA Rules not processed for user `00609e23ec8d4eadb9709b4e497810e3`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:58:54.781039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b5c4cb1f-cca9-43d2-a995-7c89a3cdd31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:54.788076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1215/2411] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:58:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:58:55.843954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f408a28-cd2b-41d0-9397-c034f11c972f service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:58:55.845268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0f408a28-cd2b-41d0-9397-c034f11c972f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:55.915668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f408a28-cd2b-41d0-9397-c034f11c972f service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:58:55.921222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9cbad-a78c-4d02-be2b-0c293b81f401 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:58:55.921863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9cbad-a78c-4d02-be2b-0c293b81f401 service placement] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:58:55.922150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9cbad-a78c-4d02-be2b-0c293b81f401 service placement] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:58:55.922431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9cbad-a78c-4d02-be2b-0c293b81f401 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:58:55.924245 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d2e9cbad-a78c-4d02-be2b-0c293b81f401 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:58:55.935363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9cbad-a78c-4d02-be2b-0c293b81f401 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:58:55.936945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9cbad-a78c-4d02-be2b-0c293b81f401 service placement] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:58:55.946599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1197/2412] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:58:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:59:04.774056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:59:05.942514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:59:07.706523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c960a15-a909-44d8-aa00-0fe2cd529dfd None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:07.707031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c960a15-a909-44d8-aa00-0fe2cd529dfd None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:07.707143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c960a15-a909-44d8-aa00-0fe2cd529dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:07.725724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2c960a15-a909-44d8-aa00-0fe2cd529dfd None None] MFA Rules not processed for user `fb838f931b7748e2baf6c2315d3cbd75`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:59:07.726950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2c960a15-a909-44d8-aa00-0fe2cd529dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:07.780922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1216/2413] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:59:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:59:14.276479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f732c0-d6dd-4259-afed-2d3fcc9c0a87 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:14.276985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f732c0-d6dd-4259-afed-2d3fcc9c0a87 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:14.277104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f732c0-d6dd-4259-afed-2d3fcc9c0a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:14.296810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d2f732c0-d6dd-4259-afed-2d3fcc9c0a87 None None] MFA Rules not processed for user `00609e23ec8d4eadb9709b4e497810e3`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:59:14.298024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d2f732c0-d6dd-4259-afed-2d3fcc9c0a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:14.305018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1198/2414] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 15:59:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:59:17.773748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=25 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:59:24.289803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:59:34.197808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798716d-6c04-41a1-8137-c1cfcf0647a1 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:34.198893 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798716d-6c04-41a1-8137-c1cfcf0647a1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:34.199022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798716d-6c04-41a1-8137-c1cfcf0647a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:34.217693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-3798716d-6c04-41a1-8137-c1cfcf0647a1 None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:59:34.218941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3798716d-6c04-41a1-8137-c1cfcf0647a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:34.278946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1217/2415] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:59:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:59:34.332140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f732c0-d6dd-4259-afed-2d3fcc9c0a87 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:59:34.334345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d2f732c0-d6dd-4259-afed-2d3fcc9c0a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:34.415026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f732c0-d6dd-4259-afed-2d3fcc9c0a87 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:59:34.419909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf468da-acf7-4866-87ba-a3808edbaa7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:59:34.420555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf468da-acf7-4866-87ba-a3808edbaa7f service swift] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:34.420860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf468da-acf7-4866-87ba-a3808edbaa7f service swift] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:34.421132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf468da-acf7-4866-87ba-a3808edbaa7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:34.429133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf468da-acf7-4866-87ba-a3808edbaa7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:59:34.430711 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf468da-acf7-4866-87ba-a3808edbaa7f service swift] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:59:34.441496 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1199/2416] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:59:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:59:43.919228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb57e49-4ea6-41d9-ac94-32fded1836ec None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:43.920110 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb57e49-4ea6-41d9-ac94-32fded1836ec None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:43.920388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb57e49-4ea6-41d9-ac94-32fded1836ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:43.939910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-bbb57e49-4ea6-41d9-ac94-32fded1836ec None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:59:43.941240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bbb57e49-4ea6-41d9-ac94-32fded1836ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:43.950984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1218/2417] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:59:43.963831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf468da-acf7-4866-87ba-a3808edbaa7f service swift] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:59:43.976068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf468da-acf7-4866-87ba-a3808edbaa7f service swift] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:59:43.981980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816268-cf47-4651-a027-d60b0571f91a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:59:43.983012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816268-cf47-4651-a027-d60b0571f91a service swift] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:43.983526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816268-cf47-4651-a027-d60b0571f91a service swift] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:43.983820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816268-cf47-4651-a027-d60b0571f91a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:43.993505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68816268-cf47-4651-a027-d60b0571f91a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:59:43.995132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68816268-cf47-4651-a027-d60b0571f91a service swift] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:59:44.005042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1200/2418] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:59:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 15:59:53.450588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb57e49-4ea6-41d9-ac94-32fded1836ec None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:59:53.453068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bbb57e49-4ea6-41d9-ac94-32fded1836ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:53.470914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb57e49-4ea6-41d9-ac94-32fded1836ec None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:59:53.475556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46f91f-af7f-40e2-9506-48da56766844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:59:53.476553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f91f-af7f-40e2-9506-48da56766844 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:53.477023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f91f-af7f-40e2-9506-48da56766844 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:53.477488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f91f-af7f-40e2-9506-48da56766844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:53.479712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ed46f91f-af7f-40e2-9506-48da56766844 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:53.579556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46f91f-af7f-40e2-9506-48da56766844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:59:53.581203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46f91f-af7f-40e2-9506-48da56766844 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:59:53.594492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1219/2419] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:59:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:59:54.000833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=38 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:59:54.676887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c63af-9a65-43f1-b13e-ad0c077bfde4 None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:54.678200 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c63af-9a65-43f1-b13e-ad0c077bfde4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:54.679078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c63af-9a65-43f1-b13e-ad0c077bfde4 None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:54.681082 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1201/2420] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:59:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:59:54.687312 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b874cd3-2820-480a-bdba-c98a884e621a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:54.687872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b874cd3-2820-480a-bdba-c98a884e621a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:54.687872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b874cd3-2820-480a-bdba-c98a884e621a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:54.713596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-8b874cd3-2820-480a-bdba-c98a884e621a None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:59:54.714871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8b874cd3-2820-480a-bdba-c98a884e621a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:54.729910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1220/2421] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:59:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:59:54.741654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c63af-9a65-43f1-b13e-ad0c077bfde4 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:59:54.743703 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4d6c63af-9a65-43f1-b13e-ad0c077bfde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:54.815530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c63af-9a65-43f1-b13e-ad0c077bfde4 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:59:54.820265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30555c9-b4f8-43ec-b6f9-57f47f5ed42d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:59:54.821168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30555c9-b4f8-43ec-b6f9-57f47f5ed42d service glance] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:54.821618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30555c9-b4f8-43ec-b6f9-57f47f5ed42d service glance] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:54.822061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30555c9-b4f8-43ec-b6f9-57f47f5ed42d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:54.834760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30555c9-b4f8-43ec-b6f9-57f47f5ed42d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:59:54.836219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30555c9-b4f8-43ec-b6f9-57f47f5ed42d service glance] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:59:54.870201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1202/2422] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:59:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:59:54.938521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681809ab-49ef-4691-bae4-c3dc93f2f02e None None] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:54.938837 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681809ab-49ef-4691-bae4-c3dc93f2f02e None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:54.939136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681809ab-49ef-4691-bae4-c3dc93f2f02e None None] PATH_INFO: `/` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:54.940489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1221/2423] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 15:59:54.945608 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab3003-b3e1-4588-bc18-e08a41e4ffd1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:54.945900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab3003-b3e1-4588-bc18-e08a41e4ffd1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:54.946183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab3003-b3e1-4588-bc18-e08a41e4ffd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:54.965829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e0ab3003-b3e1-4588-bc18-e08a41e4ffd1 None None] MFA Rules not processed for user `bf1915b9919442d692bcedf25f53320c`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:59:54.967917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e0ab3003-b3e1-4588-bc18-e08a41e4ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:54.975225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1203/2424] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:59:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:59:54.988824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681809ab-49ef-4691-bae4-c3dc93f2f02e None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:59:55.002700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681809ab-49ef-4691-bae4-c3dc93f2f02e None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:59:55.007265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92718a-1c38-41ff-b0bb-0fb0fa2a5aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:59:55.008064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92718a-1c38-41ff-b0bb-0fb0fa2a5aed service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:55.008445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92718a-1c38-41ff-b0bb-0fb0fa2a5aed service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:55.008820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92718a-1c38-41ff-b0bb-0fb0fa2a5aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:55.019006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92718a-1c38-41ff-b0bb-0fb0fa2a5aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:59:55.020575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92718a-1c38-41ff-b0bb-0fb0fa2a5aed service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:59:55.031084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1222/2425] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:59:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:59:55.123747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682101f4-ca98-486c-a7a5-3d950b6c664c None None] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:55.124747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682101f4-ca98-486c-a7a5-3d950b6c664c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:55.125029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682101f4-ca98-486c-a7a5-3d950b6c664c None None] PATH_INFO: `/` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:55.127477 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1204/2426] 10.222.0.62 () {54 vars in 921 bytes} [Wed Jun 15 15:59:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:59:55.132691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1fb09-0a3c-4cc7-8d05-d2d2990c6d33 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:55.133005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1fb09-0a3c-4cc7-8d05-d2d2990c6d33 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:55.133347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1fb09-0a3c-4cc7-8d05-d2d2990c6d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:55.153613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ebf1fb09-0a3c-4cc7-8d05-d2d2990c6d33 None None] MFA Rules not processed for user `4327e4eed5a1436ea1a00c2209bc5750`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:59:55.155832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ebf1fb09-0a3c-4cc7-8d05-d2d2990c6d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:55.166643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1223/2427] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Jun 15 15:59:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:59:55.177577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682101f4-ca98-486c-a7a5-3d950b6c664c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:59:55.190204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682101f4-ca98-486c-a7a5-3d950b6c664c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:59:55.194689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efef80d-95f0-4048-a5bd-f77beb8aecf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:59:55.195660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efef80d-95f0-4048-a5bd-f77beb8aecf5 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:55.195953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efef80d-95f0-4048-a5bd-f77beb8aecf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:55.196232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efef80d-95f0-4048-a5bd-f77beb8aecf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:55.208704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efef80d-95f0-4048-a5bd-f77beb8aecf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:59:55.210638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efef80d-95f0-4048-a5bd-f77beb8aecf5 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:59:55.225326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1205/2428] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:59:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:59:57.149115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1fb09-0a3c-4cc7-8d05-d2d2990c6d33 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:59:57.160581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1fb09-0a3c-4cc7-8d05-d2d2990c6d33 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:59:57.165704 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06b06b-f41e-40ff-bbcd-9adc848d291a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:59:57.166712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06b06b-f41e-40ff-bbcd-9adc848d291a service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:57.167071 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06b06b-f41e-40ff-bbcd-9adc848d291a service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:57.167464 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06b06b-f41e-40ff-bbcd-9adc848d291a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:57.169699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6a06b06b-f41e-40ff-bbcd-9adc848d291a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:57.189508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06b06b-f41e-40ff-bbcd-9adc848d291a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:59:57.191287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06b06b-f41e-40ff-bbcd-9adc848d291a service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:59:57.207157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1224/2429] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 15:59:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:59:58.703937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944e301-7101-475c-ab4d-0b38ba1053c9 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:58.704495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944e301-7101-475c-ab4d-0b38ba1053c9 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:58.704495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944e301-7101-475c-ab4d-0b38ba1053c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:58.728312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e944e301-7101-475c-ab4d-0b38ba1053c9 None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:59:58.729545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e944e301-7101-475c-ab4d-0b38ba1053c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:59:58.739282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1206/2430] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 15:59:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:59:58.755487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06b06b-f41e-40ff-bbcd-9adc848d291a service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:59:58.761516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=81 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:59:58.770599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06b06b-f41e-40ff-bbcd-9adc848d291a service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:59:58.777923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8b6f8-5a24-4d9c-865d-c867b11b3ca5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:59:58.778595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8b6f8-5a24-4d9c-865d-c867b11b3ca5 service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:59:58.779639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8b6f8-5a24-4d9c-865d-c867b11b3ca5 service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:59:58.780123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8b6f8-5a24-4d9c-865d-c867b11b3ca5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:59:58.790264 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8b6f8-5a24-4d9c-865d-c867b11b3ca5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:59:58.792508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8b6f8-5a24-4d9c-865d-c867b11b3ca5 service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:59:58.804290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1225/2431] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 15:59:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 16:00:01.990495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e572d4-f0ad-4c81-93c1-4c9e750b057b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:01.991164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e572d4-f0ad-4c81-93c1-4c9e750b057b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:01.991164 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e572d4-f0ad-4c81-93c1-4c9e750b057b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:02.009854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c9e572d4-f0ad-4c81-93c1-4c9e750b057b None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:02.011072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c9e572d4-f0ad-4c81-93c1-4c9e750b057b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:02.019737 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1207/2432] 10.222.0.62 () {58 vars in 967 bytes} [Wed Jun 15 16:00:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:02.032215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8b6f8-5a24-4d9c-865d-c867b11b3ca5 service swift] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:02.043814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8b6f8-5a24-4d9c-865d-c867b11b3ca5 service swift] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:02.049895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a5c08-1c9a-454e-9917-be537490980a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:02.050621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a5c08-1c9a-454e-9917-be537490980a service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:02.051019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a5c08-1c9a-454e-9917-be537490980a service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:02.051443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a5c08-1c9a-454e-9917-be537490980a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:02.061591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a5c08-1c9a-454e-9917-be537490980a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:02.063406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a5c08-1c9a-454e-9917-be537490980a service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:02.079067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1226/2433] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 16:00:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 16:00:03.474851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e572d4-f0ad-4c81-93c1-4c9e750b057b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:03.476277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c9e572d4-f0ad-4c81-93c1-4c9e750b057b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:03.492087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e572d4-f0ad-4c81-93c1-4c9e750b057b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:03.498890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a2ac8-5bea-42ca-9ab2-d062dc6fed84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:03.499936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a2ac8-5bea-42ca-9ab2-d062dc6fed84 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:03.501689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a2ac8-5bea-42ca-9ab2-d062dc6fed84 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:03.502240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a2ac8-5bea-42ca-9ab2-d062dc6fed84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:03.504660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-260a2ac8-5bea-42ca-9ab2-d062dc6fed84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:03.595005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a2ac8-5bea-42ca-9ab2-d062dc6fed84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:03.596606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a2ac8-5bea-42ca-9ab2-d062dc6fed84 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:03.615782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1208/2434] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:04.219031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a5c08-1c9a-454e-9917-be537490980a service swift] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:04.236449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a5c08-1c9a-454e-9917-be537490980a service swift] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:04.241259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:04.242134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:04.242416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:04.242698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] PATH_INFO: `/v3/users/00609e23ec8d4eadb9709b4e497810e3` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:04.244098 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00609e23ec8d4eadb9709b4e497810e3)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:04.246021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:04.294597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00609e23ec8d4eadb9709b4e497810e3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.302326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00609e23ec8d4eadb9709b4e497810e3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.309216 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00609e23ec8d4eadb9709b4e497810e3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.309839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00609e23ec8d4eadb9709b4e497810e3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.323461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-46eeab63-ba92-4c3f-8abc-62c7765ed334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00609e23ec8d4eadb9709b4e497810e3'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.330648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1227/2435] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:00:04 2022] DELETE /identity/v3/users/00609e23ec8d4eadb9709b4e497810e3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:04.535728 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a2ac8-5bea-42ca-9ab2-d062dc6fed84 service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:04.546468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a2ac8-5bea-42ca-9ab2-d062dc6fed84 service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:04.585001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:04.605417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:04.608966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:04.609479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:04.610052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] PATH_INFO: `/v3/projects/4babb3ea9898402ba7898395d59b4beb` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:04.611688 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4babb3ea9898402ba7898395d59b4beb)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:04.613678 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:04.667501 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4babb3ea9898402ba7898395d59b4beb', 'request_id': 'req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.675835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4babb3ea9898402ba7898395d59b4beb', 'request_id': 'req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.689094 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4babb3ea9898402ba7898395d59b4beb', 'request_id': 'req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.700336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] The token cache is being invalidate because project 4babb3ea9898402ba7898395d59b4beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:00:04.700626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.701296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59a35c83-2512-4a3f-b9d3-6d9bd95f716f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:04.702872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1209/2436] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:00:04 2022] DELETE /identity/v3/projects/4babb3ea9898402ba7898395d59b4beb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:04.707677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47521e58-8e75-4556-aa6f-23bd169870c1 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:04.707961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47521e58-8e75-4556-aa6f-23bd169870c1 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:04.708237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47521e58-8e75-4556-aa6f-23bd169870c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:04.725910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-47521e58-8e75-4556-aa6f-23bd169870c1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:04.727152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-47521e58-8e75-4556-aa6f-23bd169870c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:04.800461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1228/2437] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:04.813625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d197ba7-8ef0-44a5-bc27-f81d9343d560 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:04.813918 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d197ba7-8ef0-44a5-bc27-f81d9343d560 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:04.814201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d197ba7-8ef0-44a5-bc27-f81d9343d560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:04.832954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0d197ba7-8ef0-44a5-bc27-f81d9343d560 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:04.834154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0d197ba7-8ef0-44a5-bc27-f81d9343d560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:04.842635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1210/2438] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:04.846877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47521e58-8e75-4556-aa6f-23bd169870c1 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:04.856818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47521e58-8e75-4556-aa6f-23bd169870c1 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:04.862214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57e163-f91b-4225-a6cd-93be1638e100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:04.862928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57e163-f91b-4225-a6cd-93be1638e100 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:04.863214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57e163-f91b-4225-a6cd-93be1638e100 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:04.863485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57e163-f91b-4225-a6cd-93be1638e100 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:04.864207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57e163-f91b-4225-a6cd-93be1638e100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:00:04.864514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57e163-f91b-4225-a6cd-93be1638e100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:04.866182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57e163-f91b-4225-a6cd-93be1638e100 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:04.879489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1229/2439] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:00:04.887095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d197ba7-8ef0-44a5-bc27-f81d9343d560 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:04.896463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d197ba7-8ef0-44a5-bc27-f81d9343d560 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:04.908880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e4550-83d8-4647-8c2a-e4dc5cd4440d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:04.911658 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e4550-83d8-4647-8c2a-e4dc5cd4440d admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:04.912140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e4550-83d8-4647-8c2a-e4dc5cd4440d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:04.913348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e4550-83d8-4647-8c2a-e4dc5cd4440d admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:04.914235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e4550-83d8-4647-8c2a-e4dc5cd4440d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:04.916427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e4550-83d8-4647-8c2a-e4dc5cd4440d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:04.950777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1211/2440] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:00:04 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:00:04.954852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57e163-f91b-4225-a6cd-93be1638e100 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:04.964926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57e163-f91b-4225-a6cd-93be1638e100 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:05.000099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:05.030254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e29a45-0b5c-43cf-a5a5-d99582ce32a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:05.031367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e29a45-0b5c-43cf-a5a5-d99582ce32a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:05.031907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e29a45-0b5c-43cf-a5a5-d99582ce32a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:05.032418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e29a45-0b5c-43cf-a5a5-d99582ce32a3 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:05.033884 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e29a45-0b5c-43cf-a5a5-d99582ce32a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:05.036629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e29a45-0b5c-43cf-a5a5-d99582ce32a3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:05.066720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1230/2441] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:00:04 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:00:05.067949 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e4550-83d8-4647-8c2a-e4dc5cd4440d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:05.078790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e4550-83d8-4647-8c2a-e4dc5cd4440d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:05.083413 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bec05-919a-4926-bc38-2d79d55da60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:05.084075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bec05-919a-4926-bc38-2d79d55da60e admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:05.084364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bec05-919a-4926-bc38-2d79d55da60e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:05.084677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bec05-919a-4926-bc38-2d79d55da60e admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:05.085446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bec05-919a-4926-bc38-2d79d55da60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:05.087248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bec05-919a-4926-bc38-2d79d55da60e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:05.098259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1212/2442] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:00:05.102172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e29a45-0b5c-43cf-a5a5-d99582ce32a3 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:05.112726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e29a45-0b5c-43cf-a5a5-d99582ce32a3 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:05.117394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8573cd-0e21-4e4a-ac93-07dac3633d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:05.118195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8573cd-0e21-4e4a-ac93-07dac3633d26 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:05.118588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8573cd-0e21-4e4a-ac93-07dac3633d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:05.118994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8573cd-0e21-4e4a-ac93-07dac3633d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:05.120003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8573cd-0e21-4e4a-ac93-07dac3633d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:05.129082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8573cd-0e21-4e4a-ac93-07dac3633d26 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:05.134423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1231/2443] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:00:05.138635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bec05-919a-4926-bc38-2d79d55da60e admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:05.150020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bec05-919a-4926-bc38-2d79d55da60e admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:05.156819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad396fa3-3a0b-47af-ab5b-45110749b9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:05.158227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad396fa3-3a0b-47af-ab5b-45110749b9cb admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:05.158592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad396fa3-3a0b-47af-ab5b-45110749b9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:05.159061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad396fa3-3a0b-47af-ab5b-45110749b9cb admin admin] PATH_INFO: `/v3/projects/000dc06e269f45a68baeb23a799abc2f/users/ebe6d55988bd4d4686c6ed8d31463b85/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:05.177558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad396fa3-3a0b-47af-ab5b-45110749b9cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=000dc06e269f45a68baeb23a799abc2f, user_id=ebe6d55988bd4d4686c6ed8d31463b85, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:05.180270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad396fa3-3a0b-47af-ab5b-45110749b9cb admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:05.193625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1213/2444] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:00:05 2022] PUT /identity/v3/projects/000dc06e269f45a68baeb23a799abc2f/users/ebe6d55988bd4d4686c6ed8d31463b85/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:05.204492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639d05a-21ce-49df-974e-ba8a9a594ba2 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:05.204786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639d05a-21ce-49df-974e-ba8a9a594ba2 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:05.205070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639d05a-21ce-49df-974e-ba8a9a594ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:05.237970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-7639d05a-21ce-49df-974e-ba8a9a594ba2 None None] MFA Rules not processed for user `ebe6d55988bd4d4686c6ed8d31463b85`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:05.239231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7639d05a-21ce-49df-974e-ba8a9a594ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:05.312795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1232/2445] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:05.333387 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad396fa3-3a0b-47af-ab5b-45110749b9cb admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:05.335112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ad396fa3-3a0b-47af-ab5b-45110749b9cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:05.391431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad396fa3-3a0b-47af-ab5b-45110749b9cb admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:05.395099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49038a8-d316-4d1d-afbf-4087d982af27 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:05.395744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49038a8-d316-4d1d-afbf-4087d982af27 service glance] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:05.396023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49038a8-d316-4d1d-afbf-4087d982af27 service glance] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:05.396296 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49038a8-d316-4d1d-afbf-4087d982af27 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:05.405727 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49038a8-d316-4d1d-afbf-4087d982af27 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:05.407196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49038a8-d316-4d1d-afbf-4087d982af27 service glance] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:05.417049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1214/2446] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:00:15.309818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:15.413085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:17.408286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639d05a-21ce-49df-974e-ba8a9a594ba2 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:17.411090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7639d05a-21ce-49df-974e-ba8a9a594ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:17.480277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639d05a-21ce-49df-974e-ba8a9a594ba2 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:17.484442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:17.485146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:17.485445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:17.485714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:17.488560 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:17.499447 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:17.500888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:17.509049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1233/2447] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:23.491015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a483a-ceb5-41ce-9548-e74e1111197f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:23.491480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a483a-ceb5-41ce-9548-e74e1111197f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:23.491581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a483a-ceb5-41ce-9548-e74e1111197f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:23.508868 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-d13a483a-ceb5-41ce-9548-e74e1111197f None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:23.510054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d13a483a-ceb5-41ce-9548-e74e1111197f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:23.564004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1215/2448] 10.222.0.62 () {58 vars in 967 bytes} [Wed Jun 15 16:00:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:23.575703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:23.577073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:23.647367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a03fb3-613f-411b-98f1-1158eab461d2 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:23.652410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767de70-6925-4d90-a8dd-9eda5f4e8392 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:23.653060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767de70-6925-4d90-a8dd-9eda5f4e8392 service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:23.653344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767de70-6925-4d90-a8dd-9eda5f4e8392 service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:23.653623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767de70-6925-4d90-a8dd-9eda5f4e8392 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:23.662785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767de70-6925-4d90-a8dd-9eda5f4e8392 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:23.664382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767de70-6925-4d90-a8dd-9eda5f4e8392 service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:23.672829 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1234/2449] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 16:00:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 16:00:33.557462 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=25 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:33.669786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=58 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:42.728207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a483a-ceb5-41ce-9548-e74e1111197f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:42.737373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a483a-ceb5-41ce-9548-e74e1111197f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:42.741389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209f05d-2083-4def-a1f9-b9620e33e69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:42.742017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209f05d-2083-4def-a1f9-b9620e33e69b service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:42.742297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209f05d-2083-4def-a1f9-b9620e33e69b service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:42.742571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209f05d-2083-4def-a1f9-b9620e33e69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:42.744483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e209f05d-2083-4def-a1f9-b9620e33e69b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:42.805728 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209f05d-2083-4def-a1f9-b9620e33e69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:42.807174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209f05d-2083-4def-a1f9-b9620e33e69b service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:42.816549 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1216/2450] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:43.651477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767de70-6925-4d90-a8dd-9eda5f4e8392 service swift] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:43.671447 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767de70-6925-4d90-a8dd-9eda5f4e8392 service swift] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:43.679183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:43.680381 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:43.680662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:43.680938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] PATH_INFO: `/v3/users/ebe6d55988bd4d4686c6ed8d31463b85` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:43.682262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebe6d55988bd4d4686c6ed8d31463b85)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:43.684344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:43.753254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebe6d55988bd4d4686c6ed8d31463b85'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:43.762055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebe6d55988bd4d4686c6ed8d31463b85'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:43.769849 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebe6d55988bd4d4686c6ed8d31463b85'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:43.770708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebe6d55988bd4d4686c6ed8d31463b85'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:43.786204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebe6d55988bd4d4686c6ed8d31463b85'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:43.794143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1235/2451] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:00:43 2022] DELETE /identity/v3/users/ebe6d55988bd4d4686c6ed8d31463b85 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:43.798375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a7b72-7874-4239-a8ae-a465e3b139ef None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:43.798901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a7b72-7874-4239-a8ae-a465e3b139ef None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:43.799007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a7b72-7874-4239-a8ae-a465e3b139ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:43.818948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f10a7b72-7874-4239-a8ae-a465e3b139ef None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:43.821113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f10a7b72-7874-4239-a8ae-a465e3b139ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:43.906168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1217/2452] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:00:43.916514 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:43.918108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:43.979332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e566141-0255-4b1a-bbf1-093cfb206de3 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:43.983008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cfdb2-c138-46df-9d04-f6d8293bf44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:43.983661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cfdb2-c138-46df-9d04-f6d8293bf44e service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:43.983945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cfdb2-c138-46df-9d04-f6d8293bf44e service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:43.984221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cfdb2-c138-46df-9d04-f6d8293bf44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:43.993912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cfdb2-c138-46df-9d04-f6d8293bf44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:43.995398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cfdb2-c138-46df-9d04-f6d8293bf44e service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:44.005081 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1236/2453] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:44.289722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a7b72-7874-4239-a8ae-a465e3b139ef None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:44.291228 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f10a7b72-7874-4239-a8ae-a465e3b139ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:44.305906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a7b72-7874-4239-a8ae-a465e3b139ef None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:44.328994 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:44.331274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.331619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.332123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] PATH_INFO: `/v3/projects/000dc06e269f45a68baeb23a799abc2f` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.333643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=000dc06e269f45a68baeb23a799abc2f)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:44.335952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:44.398041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '000dc06e269f45a68baeb23a799abc2f', 'request_id': 'req-ca46d7a9-f51b-4888-be4e-b109911c2154'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:44.405565 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '000dc06e269f45a68baeb23a799abc2f', 'request_id': 'req-ca46d7a9-f51b-4888-be4e-b109911c2154'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:44.414559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '000dc06e269f45a68baeb23a799abc2f', 'request_id': 'req-ca46d7a9-f51b-4888-be4e-b109911c2154'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:44.426826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] The token cache is being invalidate because project 000dc06e269f45a68baeb23a799abc2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:00:44.427011 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:44.427674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ca46d7a9-f51b-4888-be4e-b109911c2154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:44.430276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1218/2454] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:00:44 2022] DELETE /identity/v3/projects/000dc06e269f45a68baeb23a799abc2f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:44.434768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4616455-f6b9-43ff-81f9-9cb2ac8a35a3 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.435228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4616455-f6b9-43ff-81f9-9cb2ac8a35a3 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.435538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4616455-f6b9-43ff-81f9-9cb2ac8a35a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.455313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b4616455-f6b9-43ff-81f9-9cb2ac8a35a3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:44.456566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b4616455-f6b9-43ff-81f9-9cb2ac8a35a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:44.521553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1237/2455] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:44.528928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29878b-c2d8-4495-bbec-91931392f2fe None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.529251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29878b-c2d8-4495-bbec-91931392f2fe None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.529515 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29878b-c2d8-4495-bbec-91931392f2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.550433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-fd29878b-c2d8-4495-bbec-91931392f2fe None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:44.551763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fd29878b-c2d8-4495-bbec-91931392f2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:44.560469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1219/2456] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:44.564975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4616455-f6b9-43ff-81f9-9cb2ac8a35a3 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:44.575879 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4616455-f6b9-43ff-81f9-9cb2ac8a35a3 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:44.581693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a83d3-63c7-447f-8294-24286a4939a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:44.582665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a83d3-63c7-447f-8294-24286a4939a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.583212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a83d3-63c7-447f-8294-24286a4939a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.583806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a83d3-63c7-447f-8294-24286a4939a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.584886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a83d3-63c7-447f-8294-24286a4939a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:00:44.585577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a83d3-63c7-447f-8294-24286a4939a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:44.587628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a83d3-63c7-447f-8294-24286a4939a1 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:44.600603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1238/2457] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:00:44.605041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29878b-c2d8-4495-bbec-91931392f2fe None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:44.614528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29878b-c2d8-4495-bbec-91931392f2fe None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:44.620639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd939162-7df2-402c-81de-fd781c3a34e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:44.621932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd939162-7df2-402c-81de-fd781c3a34e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.622227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd939162-7df2-402c-81de-fd781c3a34e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.622507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd939162-7df2-402c-81de-fd781c3a34e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.623240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd939162-7df2-402c-81de-fd781c3a34e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:44.625148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd939162-7df2-402c-81de-fd781c3a34e3 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:44.635216 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:44.661961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1220/2458] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:00:44 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:00:44.663679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a83d3-63c7-447f-8294-24286a4939a1 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:44.669282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:44.674941 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a83d3-63c7-447f-8294-24286a4939a1 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:44.753876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d7ea8-3ad5-4561-aec5-a60777337ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:44.754590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d7ea8-3ad5-4561-aec5-a60777337ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.754890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d7ea8-3ad5-4561-aec5-a60777337ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.755171 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d7ea8-3ad5-4561-aec5-a60777337ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.756008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d7ea8-3ad5-4561-aec5-a60777337ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:44.757978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d7ea8-3ad5-4561-aec5-a60777337ee3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:44.780065 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1239/2459] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:00:44 2022] POST /identity/v3/users => generated 282 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:00:44.784170 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd939162-7df2-402c-81de-fd781c3a34e3 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:44.794973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd939162-7df2-402c-81de-fd781c3a34e3 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:44.799600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979474c-0fb8-4074-853c-6053582f0992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:44.800262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979474c-0fb8-4074-853c-6053582f0992 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.800547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979474c-0fb8-4074-853c-6053582f0992 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.800825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979474c-0fb8-4074-853c-6053582f0992 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.801555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979474c-0fb8-4074-853c-6053582f0992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:44.803301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979474c-0fb8-4074-853c-6053582f0992 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:44.813718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1221/2460] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:00:44.817927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d7ea8-3ad5-4561-aec5-a60777337ee3 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:44.828066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d7ea8-3ad5-4561-aec5-a60777337ee3 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:44.832695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83ec09-ac17-483a-8232-ab83fd2c80aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:44.833355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83ec09-ac17-483a-8232-ab83fd2c80aa admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.833637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83ec09-ac17-483a-8232-ab83fd2c80aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.833910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83ec09-ac17-483a-8232-ab83fd2c80aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.835195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83ec09-ac17-483a-8232-ab83fd2c80aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:44.836821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83ec09-ac17-483a-8232-ab83fd2c80aa admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:44.847061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1240/2461] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:00:44.851752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979474c-0fb8-4074-853c-6053582f0992 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:44.861682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979474c-0fb8-4074-853c-6053582f0992 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:44.866347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fefc56-c465-4d03-86a3-6e1cf8ec23ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:44.867265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fefc56-c465-4d03-86a3-6e1cf8ec23ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.867553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fefc56-c465-4d03-86a3-6e1cf8ec23ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.867834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fefc56-c465-4d03-86a3-6e1cf8ec23ea admin admin] PATH_INFO: `/v3/projects/d238672f4f014ad89541209711b8c0bb/users/4582a193951844fb902a088ee77ac112/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.882592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fefc56-c465-4d03-86a3-6e1cf8ec23ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d238672f4f014ad89541209711b8c0bb, user_id=4582a193951844fb902a088ee77ac112, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:44.885284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fefc56-c465-4d03-86a3-6e1cf8ec23ea admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:44.900214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1222/2462] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:00:44 2022] PUT /identity/v3/projects/d238672f4f014ad89541209711b8c0bb/users/4582a193951844fb902a088ee77ac112/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:44.910266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1aabe-97da-4ab5-a091-f352921a36c4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:44.910564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1aabe-97da-4ab5-a091-f352921a36c4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:44.910909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1aabe-97da-4ab5-a091-f352921a36c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:44.939933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-40d1aabe-97da-4ab5-a091-f352921a36c4 None None] MFA Rules not processed for user `4582a193951844fb902a088ee77ac112`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:44.941175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-40d1aabe-97da-4ab5-a091-f352921a36c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:45.022537 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1241/2463] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:45.034185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fefc56-c465-4d03-86a3-6e1cf8ec23ea admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:45.035958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-01fefc56-c465-4d03-86a3-6e1cf8ec23ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:45.101844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fefc56-c465-4d03-86a3-6e1cf8ec23ea admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:45.105939 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d13df-ff96-4b3e-97b4-c13615d45fd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:45.106689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d13df-ff96-4b3e-97b4-c13615d45fd0 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:45.107071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d13df-ff96-4b3e-97b4-c13615d45fd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:45.107719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d13df-ff96-4b3e-97b4-c13615d45fd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:45.117360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d13df-ff96-4b3e-97b4-c13615d45fd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:45.119217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d13df-ff96-4b3e-97b4-c13615d45fd0 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:45.133726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1223/2464] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:45 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:45.761667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1aabe-97da-4ab5-a091-f352921a36c4 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:45.764832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-40d1aabe-97da-4ab5-a091-f352921a36c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:45.836345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d13df-ff96-4b3e-97b4-c13615d45fd0 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:45.841653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-203d13df-ff96-4b3e-97b4-c13615d45fd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:45.882923 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1aabe-97da-4ab5-a091-f352921a36c4 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:45.888497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0f88f-e84d-4779-9902-64acfa77869c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:45.889179 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f88f-e84d-4779-9902-64acfa77869c service placement] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:45.889460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f88f-e84d-4779-9902-64acfa77869c service placement] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:45.889741 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f88f-e84d-4779-9902-64acfa77869c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:45.891593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-08b0f88f-e84d-4779-9902-64acfa77869c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:45.903927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0f88f-e84d-4779-9902-64acfa77869c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:45.905510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0f88f-e84d-4779-9902-64acfa77869c service placement] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:45.910613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d13df-ff96-4b3e-97b4-c13615d45fd0 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:45.911109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:45.915904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:45.916711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1242/2465] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:45.916839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:45.917042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:45.917552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] PATH_INFO: `/v3/users/434c81b82dee404f8d143f1bdf56d3ca` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:45.934218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434c81b82dee404f8d143f1bdf56d3ca)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:45.936360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:45.983365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434c81b82dee404f8d143f1bdf56d3ca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:45.985066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:45.992359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434c81b82dee404f8d143f1bdf56d3ca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:46.000629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434c81b82dee404f8d143f1bdf56d3ca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:46.001263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434c81b82dee404f8d143f1bdf56d3ca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:46.001855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-684f217a-1571-4948-9815-c0825c04e4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434c81b82dee404f8d143f1bdf56d3ca'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:46.018838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1224/2466] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:00:45 2022] DELETE /identity/v3/users/434c81b82dee404f8d143f1bdf56d3ca => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:00:46.197731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0f88f-e84d-4779-9902-64acfa77869c service placement] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:46.216647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0f88f-e84d-4779-9902-64acfa77869c service placement] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:46.274928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:46.275725 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.276006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.276286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] PATH_INFO: `/v3/projects/669dc76c39b3427b8c55fd47de62595f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.286713 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=669dc76c39b3427b8c55fd47de62595f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:46.288680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:46.334112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '669dc76c39b3427b8c55fd47de62595f', 'request_id': 'req-10b2cbd7-65dc-4531-9e43-836d3c996138'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:46.341980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '669dc76c39b3427b8c55fd47de62595f', 'request_id': 'req-10b2cbd7-65dc-4531-9e43-836d3c996138'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:46.348995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '669dc76c39b3427b8c55fd47de62595f', 'request_id': 'req-10b2cbd7-65dc-4531-9e43-836d3c996138'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:46.359103 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] The token cache is being invalidate because project 669dc76c39b3427b8c55fd47de62595f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:00:46.359389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:46.360001 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-10b2cbd7-65dc-4531-9e43-836d3c996138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:46.361495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1243/2467] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:00:46 2022] DELETE /identity/v3/projects/669dc76c39b3427b8c55fd47de62595f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:46.366954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390472a-0613-48dd-9416-7601347d471d None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.367249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390472a-0613-48dd-9416-7601347d471d None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.367528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390472a-0613-48dd-9416-7601347d471d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.385594 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9390472a-0613-48dd-9416-7601347d471d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:46.390211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9390472a-0613-48dd-9416-7601347d471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:46.444572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1225/2468] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:46.452387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d620e-a8dc-4ad9-a906-ad97573c7f54 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.452676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d620e-a8dc-4ad9-a906-ad97573c7f54 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.452955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d620e-a8dc-4ad9-a906-ad97573c7f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.470563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-0b1d620e-a8dc-4ad9-a906-ad97573c7f54 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:46.471838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0b1d620e-a8dc-4ad9-a906-ad97573c7f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:46.480533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1244/2469] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:46.484777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390472a-0613-48dd-9416-7601347d471d None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:46.495203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390472a-0613-48dd-9416-7601347d471d None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:46.499956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad8631-ae40-4bef-bfaa-71e37af3ae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:46.500621 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8631-ae40-4bef-bfaa-71e37af3ae66 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.500909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8631-ae40-4bef-bfaa-71e37af3ae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.501189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8631-ae40-4bef-bfaa-71e37af3ae66 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.501913 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad8631-ae40-4bef-bfaa-71e37af3ae66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:00:46.502222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad8631-ae40-4bef-bfaa-71e37af3ae66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:46.503847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad8631-ae40-4bef-bfaa-71e37af3ae66 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:46.516293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1226/2470] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:00:46.520659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d620e-a8dc-4ad9-a906-ad97573c7f54 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:46.530848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d620e-a8dc-4ad9-a906-ad97573c7f54 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:46.535897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f80bef-5525-44ee-85e9-8bf409154066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:46.536556 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f80bef-5525-44ee-85e9-8bf409154066 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.536865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f80bef-5525-44ee-85e9-8bf409154066 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.537141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f80bef-5525-44ee-85e9-8bf409154066 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.537838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f80bef-5525-44ee-85e9-8bf409154066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:46.540067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f80bef-5525-44ee-85e9-8bf409154066 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:46.579123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1245/2471] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:00:46 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:00:46.580986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad8631-ae40-4bef-bfaa-71e37af3ae66 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:46.591166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad8631-ae40-4bef-bfaa-71e37af3ae66 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:46.653006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867223f1-83c7-435f-acb5-f0bf92b671a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:46.653675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867223f1-83c7-435f-acb5-f0bf92b671a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.653982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867223f1-83c7-435f-acb5-f0bf92b671a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.654262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867223f1-83c7-435f-acb5-f0bf92b671a2 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.654974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867223f1-83c7-435f-acb5-f0bf92b671a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:46.656867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867223f1-83c7-435f-acb5-f0bf92b671a2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:46.677947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1227/2472] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:00:46 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:00:46.682908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f80bef-5525-44ee-85e9-8bf409154066 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:46.696266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f80bef-5525-44ee-85e9-8bf409154066 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:46.699208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0154535-7671-4a33-ac43-cf790fb592c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:46.699869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154535-7671-4a33-ac43-cf790fb592c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.700153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154535-7671-4a33-ac43-cf790fb592c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.700435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0154535-7671-4a33-ac43-cf790fb592c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.701160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0154535-7671-4a33-ac43-cf790fb592c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:46.702788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0154535-7671-4a33-ac43-cf790fb592c2 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:46.713009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1246/2473] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:00:46.717026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867223f1-83c7-435f-acb5-f0bf92b671a2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:46.727516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867223f1-83c7-435f-acb5-f0bf92b671a2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:46.732474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864d780-3d66-4e0c-a19d-29a4db2d9346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:46.733240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864d780-3d66-4e0c-a19d-29a4db2d9346 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.733534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864d780-3d66-4e0c-a19d-29a4db2d9346 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.733818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864d780-3d66-4e0c-a19d-29a4db2d9346 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.734575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864d780-3d66-4e0c-a19d-29a4db2d9346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:46.736210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864d780-3d66-4e0c-a19d-29a4db2d9346 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:46.750349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1228/2474] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:00:46.752019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0154535-7671-4a33-ac43-cf790fb592c2 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:46.761912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0154535-7671-4a33-ac43-cf790fb592c2 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:46.766622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7e46d-8ee9-4798-8d62-54261efe1b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:46.767525 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7e46d-8ee9-4798-8d62-54261efe1b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.767811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7e46d-8ee9-4798-8d62-54261efe1b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.768092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7e46d-8ee9-4798-8d62-54261efe1b66 admin admin] PATH_INFO: `/v3/projects/333966f6f9da41e3b940ef9f253913c5/users/07b16e07166f48db9070ee57f57185e2/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.782712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7e46d-8ee9-4798-8d62-54261efe1b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=333966f6f9da41e3b940ef9f253913c5, user_id=07b16e07166f48db9070ee57f57185e2, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:46.785596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7e46d-8ee9-4798-8d62-54261efe1b66 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:46.799231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1247/2475] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:00:46 2022] PUT /identity/v3/projects/333966f6f9da41e3b940ef9f253913c5/users/07b16e07166f48db9070ee57f57185e2/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:46.810790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f3239-8dfa-4f90-ab94-ab391b5ede2b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:46.811090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f3239-8dfa-4f90-ab94-ab391b5ede2b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:46.811373 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f3239-8dfa-4f90-ab94-ab391b5ede2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:46.837906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-237f3239-8dfa-4f90-ab94-ab391b5ede2b None None] MFA Rules not processed for user `07b16e07166f48db9070ee57f57185e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:46.839178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-237f3239-8dfa-4f90-ab94-ab391b5ede2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:46.858098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:46.910803 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1229/2476] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:00:46.924826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7e46d-8ee9-4798-8d62-54261efe1b66 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:46.926790 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-13b7e46d-8ee9-4798-8d62-54261efe1b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:46.963112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:47.030678 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7e46d-8ee9-4798-8d62-54261efe1b66 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:47.038383 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f74703-3cd0-4ced-bc3a-00ded1714e68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:47.039089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f74703-3cd0-4ced-bc3a-00ded1714e68 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:47.039370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f74703-3cd0-4ced-bc3a-00ded1714e68 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:47.039645 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f74703-3cd0-4ced-bc3a-00ded1714e68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:47.047739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f74703-3cd0-4ced-bc3a-00ded1714e68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:47.049527 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f74703-3cd0-4ced-bc3a-00ded1714e68 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:47.058608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1248/2477] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:55.345379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f3239-8dfa-4f90-ab94-ab391b5ede2b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:55.405754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f3239-8dfa-4f90-ab94-ab391b5ede2b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:55.410500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:55.411388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:55.411676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:55.411957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] PATH_INFO: `/v3/users/07b16e07166f48db9070ee57f57185e2` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:55.413648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07b16e07166f48db9070ee57f57185e2)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:55.415596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:55.463045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07b16e07166f48db9070ee57f57185e2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:55.470127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07b16e07166f48db9070ee57f57185e2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:55.477753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07b16e07166f48db9070ee57f57185e2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:55.478596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07b16e07166f48db9070ee57f57185e2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:55.479235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07b16e07166f48db9070ee57f57185e2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:55.497234 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1230/2478] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:00:55 2022] DELETE /identity/v3/users/07b16e07166f48db9070ee57f57185e2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:55.501776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c00864-cc39-4201-bae8-5b156363e826 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:55.502246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c00864-cc39-4201-bae8-5b156363e826 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:55.502357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c00864-cc39-4201-bae8-5b156363e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:55.518930 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b6c00864-cc39-4201-bae8-5b156363e826 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:55.520164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b6c00864-cc39-4201-bae8-5b156363e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:55.578807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1249/2479] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:55.588810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:55.590209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:55.633777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:55.649886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca8287-ea4a-4a04-8466-a014cc39be75 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:55.653535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574348c-7d91-44b2-b453-96489f52bc36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:55.654175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574348c-7d91-44b2-b453-96489f52bc36 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:55.654466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574348c-7d91-44b2-b453-96489f52bc36 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:55.654771 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574348c-7d91-44b2-b453-96489f52bc36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:55.664023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574348c-7d91-44b2-b453-96489f52bc36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:55.665451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574348c-7d91-44b2-b453-96489f52bc36 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:55.675203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1231/2480] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:55.936712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c00864-cc39-4201-bae8-5b156363e826 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:55.938219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b6c00864-cc39-4201-bae8-5b156363e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:55.951721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c00864-cc39-4201-bae8-5b156363e826 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:55.956391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:55.957242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:55.957518 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:55.957791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] PATH_INFO: `/v3/projects/333966f6f9da41e3b940ef9f253913c5` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:55.958986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=333966f6f9da41e3b940ef9f253913c5)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:55.960901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.007472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333966f6f9da41e3b940ef9f253913c5', 'request_id': 'req-452aef8c-751b-4964-9d6d-a31eaba03acb'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:56.013437 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333966f6f9da41e3b940ef9f253913c5', 'request_id': 'req-452aef8c-751b-4964-9d6d-a31eaba03acb'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:56.021961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333966f6f9da41e3b940ef9f253913c5', 'request_id': 'req-452aef8c-751b-4964-9d6d-a31eaba03acb'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:56.032561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] The token cache is being invalidate because project 333966f6f9da41e3b940ef9f253913c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:00:56.032853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:56.033477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-452aef8c-751b-4964-9d6d-a31eaba03acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:56.035301 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1250/2481] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:00:55 2022] DELETE /identity/v3/projects/333966f6f9da41e3b940ef9f253913c5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:56.041258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152670b-cd3a-4fa4-a652-3a74f5853f6b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.041558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152670b-cd3a-4fa4-a652-3a74f5853f6b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.041840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152670b-cd3a-4fa4-a652-3a74f5853f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.060093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-9152670b-cd3a-4fa4-a652-3a74f5853f6b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:56.061318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-9152670b-cd3a-4fa4-a652-3a74f5853f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:56.123067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1232/2482] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:56.130481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738b067-d8ba-41c8-96b8-8958467992b6 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.130845 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738b067-d8ba-41c8-96b8-8958467992b6 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.131220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738b067-d8ba-41c8-96b8-8958467992b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.149427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-4738b067-d8ba-41c8-96b8-8958467992b6 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:56.150629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4738b067-d8ba-41c8-96b8-8958467992b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:56.159856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1251/2483] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:56.164648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152670b-cd3a-4fa4-a652-3a74f5853f6b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.174270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152670b-cd3a-4fa4-a652-3a74f5853f6b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.179502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e1a61-b902-465c-a873-f21dc1a92a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.180162 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e1a61-b902-465c-a873-f21dc1a92a4c admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.180449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e1a61-b902-465c-a873-f21dc1a92a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.180730 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e1a61-b902-465c-a873-f21dc1a92a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.181482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e1a61-b902-465c-a873-f21dc1a92a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:00:56.181792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e1a61-b902-465c-a873-f21dc1a92a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.183423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e1a61-b902-465c-a873-f21dc1a92a4c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.195029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1233/2484] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:00:56.199157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738b067-d8ba-41c8-96b8-8958467992b6 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.208285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738b067-d8ba-41c8-96b8-8958467992b6 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.212994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83ee16-9c84-4db8-a523-1a247ad492ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.213644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83ee16-9c84-4db8-a523-1a247ad492ab admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.213927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83ee16-9c84-4db8-a523-1a247ad492ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.214207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83ee16-9c84-4db8-a523-1a247ad492ab admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.214958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83ee16-9c84-4db8-a523-1a247ad492ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.216856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83ee16-9c84-4db8-a523-1a247ad492ab admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.235642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:56.244729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1252/2485] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:00:56 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:00:56.248470 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e1a61-b902-465c-a873-f21dc1a92a4c admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.259718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e1a61-b902-465c-a873-f21dc1a92a4c admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.323779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717e32d-d83b-430a-8407-f205a4162faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.324483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717e32d-d83b-430a-8407-f205a4162faa admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.324768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717e32d-d83b-430a-8407-f205a4162faa admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.325047 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717e32d-d83b-430a-8407-f205a4162faa admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.325770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717e32d-d83b-430a-8407-f205a4162faa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.327726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717e32d-d83b-430a-8407-f205a4162faa admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.347045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1234/2486] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:00:56 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:00:56.351633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83ee16-9c84-4db8-a523-1a247ad492ab admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.361908 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83ee16-9c84-4db8-a523-1a247ad492ab admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.367251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2095f5a-11d3-467d-8e54-ecb208a4a139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.367898 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2095f5a-11d3-467d-8e54-ecb208a4a139 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.368230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2095f5a-11d3-467d-8e54-ecb208a4a139 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.368448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2095f5a-11d3-467d-8e54-ecb208a4a139 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.369164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2095f5a-11d3-467d-8e54-ecb208a4a139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.370784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2095f5a-11d3-467d-8e54-ecb208a4a139 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.380926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1253/2487] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:00:56.384666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717e32d-d83b-430a-8407-f205a4162faa admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.394434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717e32d-d83b-430a-8407-f205a4162faa admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.399576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57accd-2bbc-43ca-a57a-b8dc325691a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.400220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57accd-2bbc-43ca-a57a-b8dc325691a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.400503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57accd-2bbc-43ca-a57a-b8dc325691a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.400785 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57accd-2bbc-43ca-a57a-b8dc325691a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.401502 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57accd-2bbc-43ca-a57a-b8dc325691a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.403106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57accd-2bbc-43ca-a57a-b8dc325691a9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.413841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1235/2488] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:00:56.417582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2095f5a-11d3-467d-8e54-ecb208a4a139 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.427403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2095f5a-11d3-467d-8e54-ecb208a4a139 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.431951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de377bfd-92be-4acc-a4f0-26fbb21670b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.432871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377bfd-92be-4acc-a4f0-26fbb21670b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.433153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377bfd-92be-4acc-a4f0-26fbb21670b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.433442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377bfd-92be-4acc-a4f0-26fbb21670b4 admin admin] PATH_INFO: `/v3/projects/61c9000ffe9445ce9ad78e80f35842c7/users/6ae87d6d375f45fd828412c9f522a90f/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.448384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de377bfd-92be-4acc-a4f0-26fbb21670b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61c9000ffe9445ce9ad78e80f35842c7, user_id=6ae87d6d375f45fd828412c9f522a90f, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.451132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de377bfd-92be-4acc-a4f0-26fbb21670b4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.464885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1254/2489] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:00:56 2022] PUT /identity/v3/projects/61c9000ffe9445ce9ad78e80f35842c7/users/6ae87d6d375f45fd828412c9f522a90f/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:00:56.475906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77855a54-d3d3-41b5-a9e3-f99a5b914f44 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.476202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77855a54-d3d3-41b5-a9e3-f99a5b914f44 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.476488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77855a54-d3d3-41b5-a9e3-f99a5b914f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.502290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-77855a54-d3d3-41b5-a9e3-f99a5b914f44 None None] MFA Rules not processed for user `6ae87d6d375f45fd828412c9f522a90f`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:56.503528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-77855a54-d3d3-41b5-a9e3-f99a5b914f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:56.541467 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:56.568765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1236/2490] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:56.573205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de377bfd-92be-4acc-a4f0-26fbb21670b4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.634070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de377bfd-92be-4acc-a4f0-26fbb21670b4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.638851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03d241-40a6-406e-b2b8-03dff39a0436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.639587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03d241-40a6-406e-b2b8-03dff39a0436 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.639868 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03d241-40a6-406e-b2b8-03dff39a0436 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.640144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03d241-40a6-406e-b2b8-03dff39a0436 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.640832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03d241-40a6-406e-b2b8-03dff39a0436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.642846 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03d241-40a6-406e-b2b8-03dff39a0436 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.668713 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1255/2491] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:00:56 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:00:56.672849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77855a54-d3d3-41b5-a9e3-f99a5b914f44 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.683520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77855a54-d3d3-41b5-a9e3-f99a5b914f44 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.738670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8fbfc-7215-4c19-b472-239a31159f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.739363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8fbfc-7215-4c19-b472-239a31159f06 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.739652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8fbfc-7215-4c19-b472-239a31159f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.739930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8fbfc-7215-4c19-b472-239a31159f06 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.740629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8fbfc-7215-4c19-b472-239a31159f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.742567 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8fbfc-7215-4c19-b472-239a31159f06 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.762655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1237/2492] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:00:56 2022] POST /identity/v3/users => generated 308 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:00:56.766553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03d241-40a6-406e-b2b8-03dff39a0436 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.782869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03d241-40a6-406e-b2b8-03dff39a0436 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.787912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48440f01-568b-4a57-8b43-b1d67c88c417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.788568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48440f01-568b-4a57-8b43-b1d67c88c417 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.788856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48440f01-568b-4a57-8b43-b1d67c88c417 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.789136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48440f01-568b-4a57-8b43-b1d67c88c417 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.789858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48440f01-568b-4a57-8b43-b1d67c88c417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.791499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48440f01-568b-4a57-8b43-b1d67c88c417 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.802019 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1256/2493] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:00:56.806004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8fbfc-7215-4c19-b472-239a31159f06 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.815798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8fbfc-7215-4c19-b472-239a31159f06 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.820959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4637ba5-7743-4694-8fff-12cadfc7b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.821834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4637ba5-7743-4694-8fff-12cadfc7b7d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.822119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4637ba5-7743-4694-8fff-12cadfc7b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.822397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4637ba5-7743-4694-8fff-12cadfc7b7d9 admin admin] PATH_INFO: `/v3/projects/f350caeff9604ff98081ed086f92b63d/users/7c671f6647654afe9396d79047d13c68/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.837331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4637ba5-7743-4694-8fff-12cadfc7b7d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f350caeff9604ff98081ed086f92b63d, user_id=7c671f6647654afe9396d79047d13c68, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.840081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4637ba5-7743-4694-8fff-12cadfc7b7d9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.853109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1238/2494] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:00:56 2022] PUT /identity/v3/projects/f350caeff9604ff98081ed086f92b63d/users/7c671f6647654afe9396d79047d13c68/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:56.856932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48440f01-568b-4a57-8b43-b1d67c88c417 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.866969 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48440f01-568b-4a57-8b43-b1d67c88c417 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.922324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102956bd-6fc3-4861-8dc0-ed5617f0ad2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.922951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102956bd-6fc3-4861-8dc0-ed5617f0ad2f admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.923239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102956bd-6fc3-4861-8dc0-ed5617f0ad2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.923519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102956bd-6fc3-4861-8dc0-ed5617f0ad2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.924231 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102956bd-6fc3-4861-8dc0-ed5617f0ad2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.925833 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102956bd-6fc3-4861-8dc0-ed5617f0ad2f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.935626 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1257/2495] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:00:56.939590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4637ba5-7743-4694-8fff-12cadfc7b7d9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:56.950217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4637ba5-7743-4694-8fff-12cadfc7b7d9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:56.954847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae3de3-cdf9-4527-922d-42bc35f2de73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:56.955724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae3de3-cdf9-4527-922d-42bc35f2de73 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.956006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae3de3-cdf9-4527-922d-42bc35f2de73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.956281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae3de3-cdf9-4527-922d-42bc35f2de73 admin admin] PATH_INFO: `/v3/projects/f350caeff9604ff98081ed086f92b63d/users/7c671f6647654afe9396d79047d13c68/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:56.958258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae3de3-cdf9-4527-922d-42bc35f2de73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f350caeff9604ff98081ed086f92b63d, user_id=7c671f6647654afe9396d79047d13c68, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:56.961445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae3de3-cdf9-4527-922d-42bc35f2de73 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:56.971631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1239/2496] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:00:56 2022] PUT /identity/v3/projects/f350caeff9604ff98081ed086f92b63d/users/7c671f6647654afe9396d79047d13c68/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:56.982580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de0dc9-2494-44ea-9e11-c081bf3ceda6 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:56.982915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de0dc9-2494-44ea-9e11-c081bf3ceda6 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:56.983228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de0dc9-2494-44ea-9e11-c081bf3ceda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:57.010680 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-f7de0dc9-2494-44ea-9e11-c081bf3ceda6 None None] MFA Rules not processed for user `7c671f6647654afe9396d79047d13c68`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:57.011915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f7de0dc9-2494-44ea-9e11-c081bf3ceda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:57.031727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:57.081008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1258/2497] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:57.092078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae3de3-cdf9-4527-922d-42bc35f2de73 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:57.093822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5bae3de3-cdf9-4527-922d-42bc35f2de73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:57.159841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae3de3-cdf9-4527-922d-42bc35f2de73 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:57.164440 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ed911-b62b-4f55-820b-2cd8b2aaa51e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:57.165090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ed911-b62b-4f55-820b-2cd8b2aaa51e service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:57.165475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ed911-b62b-4f55-820b-2cd8b2aaa51e service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:57.165645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ed911-b62b-4f55-820b-2cd8b2aaa51e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:57.175274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ed911-b62b-4f55-820b-2cd8b2aaa51e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:57.176773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ed911-b62b-4f55-820b-2cd8b2aaa51e service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:57.186529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1240/2498] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:57 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:59.227469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de0dc9-2494-44ea-9e11-c081bf3ceda6 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:59.291355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de0dc9-2494-44ea-9e11-c081bf3ceda6 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:59.295961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:59.296828 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:59.297109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:59.297386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] PATH_INFO: `/v3/users/6ae87d6d375f45fd828412c9f522a90f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:59.298570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae87d6d375f45fd828412c9f522a90f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:59.300523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:59.347130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae87d6d375f45fd828412c9f522a90f'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.354686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae87d6d375f45fd828412c9f522a90f'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.361579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae87d6d375f45fd828412c9f522a90f'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.362220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae87d6d375f45fd828412c9f522a90f'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.374633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae87d6d375f45fd828412c9f522a90f'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.380441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1259/2499] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:00:59 2022] DELETE /identity/v3/users/6ae87d6d375f45fd828412c9f522a90f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:59.385008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096cee4-c0b5-4ae4-bf08-2cb961a8990a None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:59.385490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096cee4-c0b5-4ae4-bf08-2cb961a8990a None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:59.385590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096cee4-c0b5-4ae4-bf08-2cb961a8990a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:59.403229 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-5096cee4-c0b5-4ae4-bf08-2cb961a8990a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:00:59.404439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5096cee4-c0b5-4ae4-bf08-2cb961a8990a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:59.434194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:59.467753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1241/2500] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:00:59.478111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:59.479624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:59.520008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:00:59.539085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403ea9d-af47-45fc-aa37-225fa7e4d92b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:59.543141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fa620-48fa-4c25-a4cf-60c7068f89b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:59.543787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fa620-48fa-4c25-a4cf-60c7068f89b5 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:59.544069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fa620-48fa-4c25-a4cf-60c7068f89b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:59.544345 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fa620-48fa-4c25-a4cf-60c7068f89b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:59.554212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fa620-48fa-4c25-a4cf-60c7068f89b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:59.555675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fa620-48fa-4c25-a4cf-60c7068f89b5 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:59.565549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1260/2501] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:00:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:00:59.870833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096cee4-c0b5-4ae4-bf08-2cb961a8990a None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:59.872324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5096cee4-c0b5-4ae4-bf08-2cb961a8990a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:00:59.888614 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096cee4-c0b5-4ae4-bf08-2cb961a8990a None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:00:59.894339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:00:59.895099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:00:59.895383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:00:59.895664 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] PATH_INFO: `/v3/projects/61c9000ffe9445ce9ad78e80f35842c7` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:00:59.896947 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61c9000ffe9445ce9ad78e80f35842c7)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:00:59.898824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:00:59.955289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61c9000ffe9445ce9ad78e80f35842c7', 'request_id': 'req-18f40230-8e91-41f1-85a6-b9d043652f81'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.961523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61c9000ffe9445ce9ad78e80f35842c7', 'request_id': 'req-18f40230-8e91-41f1-85a6-b9d043652f81'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.969265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61c9000ffe9445ce9ad78e80f35842c7', 'request_id': 'req-18f40230-8e91-41f1-85a6-b9d043652f81'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.981241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] The token cache is being invalidate because project 61c9000ffe9445ce9ad78e80f35842c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:00:59.981697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.982677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:00:59.984701 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1242/2502] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:00:59 2022] DELETE /identity/v3/projects/61c9000ffe9445ce9ad78e80f35842c7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:00:59.989303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fa620-48fa-4c25-a4cf-60c7068f89b5 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:00:59.999946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fa620-48fa-4c25-a4cf-60c7068f89b5 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:00.064889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:00.065762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:00.066051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:00.066327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] PATH_INFO: `/v3/users/7c671f6647654afe9396d79047d13c68` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:00.067552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c671f6647654afe9396d79047d13c68)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:00.069529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:00.135486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c671f6647654afe9396d79047d13c68'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.143777 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c671f6647654afe9396d79047d13c68'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.152150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c671f6647654afe9396d79047d13c68'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.152834 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c671f6647654afe9396d79047d13c68'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.166601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8c691ac4-361f-46b8-9e5c-dd3abb3fc483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c671f6647654afe9396d79047d13c68'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.174263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1261/2503] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:00:59 2022] DELETE /identity/v3/users/7c671f6647654afe9396d79047d13c68 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:00.461603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:00.463209 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:00.479774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f40230-8e91-41f1-85a6-b9d043652f81 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:00.550862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:00.551745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:00.552215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:00.552505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] PATH_INFO: `/v3/projects/f350caeff9604ff98081ed086f92b63d` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:00.554411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f350caeff9604ff98081ed086f92b63d)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:00.556332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:00.603488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f350caeff9604ff98081ed086f92b63d', 'request_id': 'req-4227e997-5001-4b08-afc8-a3b9d82fbf76'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.609584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f350caeff9604ff98081ed086f92b63d', 'request_id': 'req-4227e997-5001-4b08-afc8-a3b9d82fbf76'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.615176 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:00.619682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f350caeff9604ff98081ed086f92b63d', 'request_id': 'req-4227e997-5001-4b08-afc8-a3b9d82fbf76'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.633000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] The token cache is being invalidate because project f350caeff9604ff98081ed086f92b63d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:01:00.633292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.633933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4227e997-5001-4b08-afc8-a3b9d82fbf76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:00.635578 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1243/2504] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:01:00 2022] DELETE /identity/v3/projects/f350caeff9604ff98081ed086f92b63d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:00.640659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452b1e5-8f70-4a9a-a511-2f13d5acc39d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:00.641127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452b1e5-8f70-4a9a-a511-2f13d5acc39d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:00.641254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452b1e5-8f70-4a9a-a511-2f13d5acc39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:00.661190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5452b1e5-8f70-4a9a-a511-2f13d5acc39d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:00.662576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5452b1e5-8f70-4a9a-a511-2f13d5acc39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:00.722536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:00.740836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1262/2505] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:00.743788 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6701382-22c3-4e96-bb4f-c52742b07fe9 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:00.744358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6701382-22c3-4e96-bb4f-c52742b07fe9 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:00.744600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6701382-22c3-4e96-bb4f-c52742b07fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:00.764974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e6701382-22c3-4e96-bb4f-c52742b07fe9 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:00.766157 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e6701382-22c3-4e96-bb4f-c52742b07fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:00.775812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1244/2506] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:00.780220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452b1e5-8f70-4a9a-a511-2f13d5acc39d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:00.792827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452b1e5-8f70-4a9a-a511-2f13d5acc39d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:00.798805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310887ed-5619-4783-bd31-09f7ec7b287b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:00.807989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310887ed-5619-4783-bd31-09f7ec7b287b admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:00.807989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310887ed-5619-4783-bd31-09f7ec7b287b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:00.807989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310887ed-5619-4783-bd31-09f7ec7b287b admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:00.807989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310887ed-5619-4783-bd31-09f7ec7b287b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:01:00.807989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310887ed-5619-4783-bd31-09f7ec7b287b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:00.807989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310887ed-5619-4783-bd31-09f7ec7b287b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:00.816825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1263/2507] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 16:01:00.821780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6701382-22c3-4e96-bb4f-c52742b07fe9 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:00.831404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6701382-22c3-4e96-bb4f-c52742b07fe9 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:00.836708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cfb02-7da0-4660-9b43-8bf2ac0c273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:00.837367 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cfb02-7da0-4660-9b43-8bf2ac0c273f admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:00.837651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cfb02-7da0-4660-9b43-8bf2ac0c273f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:00.838048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cfb02-7da0-4660-9b43-8bf2ac0c273f admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:00.838943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cfb02-7da0-4660-9b43-8bf2ac0c273f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:00.841230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cfb02-7da0-4660-9b43-8bf2ac0c273f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:00.868572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1245/2508] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:01:00 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:00.872872 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310887ed-5619-4783-bd31-09f7ec7b287b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:00.883505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310887ed-5619-4783-bd31-09f7ec7b287b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:00.939008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f795f-d78b-4c20-bda7-4b03306afb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:00.939674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f795f-d78b-4c20-bda7-4b03306afb50 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:00.939956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f795f-d78b-4c20-bda7-4b03306afb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:00.940233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f795f-d78b-4c20-bda7-4b03306afb50 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:00.940909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f795f-d78b-4c20-bda7-4b03306afb50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:00.942844 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f795f-d78b-4c20-bda7-4b03306afb50 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:00.969488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1264/2509] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:01:00 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:00.973802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cfb02-7da0-4660-9b43-8bf2ac0c273f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:00.984250 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cfb02-7da0-4660-9b43-8bf2ac0c273f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:00.988889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ef4c7-8006-40fe-9da4-0212d5a34de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:00.989544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ef4c7-8006-40fe-9da4-0212d5a34de6 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:00.989829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ef4c7-8006-40fe-9da4-0212d5a34de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:00.990110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ef4c7-8006-40fe-9da4-0212d5a34de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:00.991407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ef4c7-8006-40fe-9da4-0212d5a34de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:00.992998 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ef4c7-8006-40fe-9da4-0212d5a34de6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.005113 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1246/2510] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:01.009657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f795f-d78b-4c20-bda7-4b03306afb50 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.019826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f795f-d78b-4c20-bda7-4b03306afb50 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.024651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e684b-4b6f-4347-a455-b0db6d52d802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.025316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e684b-4b6f-4347-a455-b0db6d52d802 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.025612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e684b-4b6f-4347-a455-b0db6d52d802 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.025894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e684b-4b6f-4347-a455-b0db6d52d802 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.026623 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e684b-4b6f-4347-a455-b0db6d52d802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.028272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e684b-4b6f-4347-a455-b0db6d52d802 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.039427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1265/2511] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:01.043532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ef4c7-8006-40fe-9da4-0212d5a34de6 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.053951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ef4c7-8006-40fe-9da4-0212d5a34de6 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.058926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babac2f7-c2af-4c86-a0de-332bf850820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.059818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babac2f7-c2af-4c86-a0de-332bf850820a admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.060090 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babac2f7-c2af-4c86-a0de-332bf850820a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.060378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babac2f7-c2af-4c86-a0de-332bf850820a admin admin] PATH_INFO: `/v3/projects/4d473f097b0b47a1866a2c5ec1c4df6f/users/64274e77a3c748d19d1615f775afd94c/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.075416 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babac2f7-c2af-4c86-a0de-332bf850820a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d473f097b0b47a1866a2c5ec1c4df6f, user_id=64274e77a3c748d19d1615f775afd94c, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.078040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babac2f7-c2af-4c86-a0de-332bf850820a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.090450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1247/2512] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:01 2022] PUT /identity/v3/projects/4d473f097b0b47a1866a2c5ec1c4df6f/users/64274e77a3c748d19d1615f775afd94c/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:01.100588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523819f-815a-43eb-9546-d778bdf67d34 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.100876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523819f-815a-43eb-9546-d778bdf67d34 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.101150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523819f-815a-43eb-9546-d778bdf67d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.127090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5523819f-815a-43eb-9546-d778bdf67d34 None None] MFA Rules not processed for user `64274e77a3c748d19d1615f775afd94c`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:01.128275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5523819f-815a-43eb-9546-d778bdf67d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:01.191259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1266/2513] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:01.196284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babac2f7-c2af-4c86-a0de-332bf850820a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.256524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babac2f7-c2af-4c86-a0de-332bf850820a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.261143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb42dc-7725-4950-a9c8-314c677c7f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.261790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb42dc-7725-4950-a9c8-314c677c7f8f admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.262083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb42dc-7725-4950-a9c8-314c677c7f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.262363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb42dc-7725-4950-a9c8-314c677c7f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.263079 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb42dc-7725-4950-a9c8-314c677c7f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.264972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb42dc-7725-4950-a9c8-314c677c7f8f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.290577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1248/2514] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:01:01 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:01.294456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523819f-815a-43eb-9546-d778bdf67d34 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.305536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523819f-815a-43eb-9546-d778bdf67d34 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.353850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:01.361523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8825d9-4867-4e9a-a110-9f9ce5d01f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.362214 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8825d9-4867-4e9a-a110-9f9ce5d01f9b admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.362497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8825d9-4867-4e9a-a110-9f9ce5d01f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.362802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8825d9-4867-4e9a-a110-9f9ce5d01f9b admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.363496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8825d9-4867-4e9a-a110-9f9ce5d01f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.365404 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8825d9-4867-4e9a-a110-9f9ce5d01f9b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.385182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1267/2515] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:01:01 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:01.389143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb42dc-7725-4950-a9c8-314c677c7f8f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.399354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb42dc-7725-4950-a9c8-314c677c7f8f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.403951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e858ad4-fc00-461c-911d-37bee8de5601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.404606 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e858ad4-fc00-461c-911d-37bee8de5601 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.404892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e858ad4-fc00-461c-911d-37bee8de5601 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.405173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e858ad4-fc00-461c-911d-37bee8de5601 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.405896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e858ad4-fc00-461c-911d-37bee8de5601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.407522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e858ad4-fc00-461c-911d-37bee8de5601 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.418624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1249/2516] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:01:01.422426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8825d9-4867-4e9a-a110-9f9ce5d01f9b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.432952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8825d9-4867-4e9a-a110-9f9ce5d01f9b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.437602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32141305-fb13-474b-bf0d-6c40cf4df1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.438482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141305-fb13-474b-bf0d-6c40cf4df1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.438799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141305-fb13-474b-bf0d-6c40cf4df1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.439085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141305-fb13-474b-bf0d-6c40cf4df1b7 admin admin] PATH_INFO: `/v3/projects/8042ed2089b84312b8a5960f3f3eba08/users/c4cf42bed76b4d759a1537f14f29aad5/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.455141 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32141305-fb13-474b-bf0d-6c40cf4df1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8042ed2089b84312b8a5960f3f3eba08, user_id=c4cf42bed76b4d759a1537f14f29aad5, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.457841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32141305-fb13-474b-bf0d-6c40cf4df1b7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.470291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1268/2517] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:01 2022] PUT /identity/v3/projects/8042ed2089b84312b8a5960f3f3eba08/users/c4cf42bed76b4d759a1537f14f29aad5/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:01.473975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e858ad4-fc00-461c-911d-37bee8de5601 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.484420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e858ad4-fc00-461c-911d-37bee8de5601 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.533310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:01.541465 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7640ed-c223-4853-a81a-c24795a37090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.542204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7640ed-c223-4853-a81a-c24795a37090 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.542507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7640ed-c223-4853-a81a-c24795a37090 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.543044 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7640ed-c223-4853-a81a-c24795a37090 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.543967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7640ed-c223-4853-a81a-c24795a37090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.545745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7640ed-c223-4853-a81a-c24795a37090 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.556130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1250/2518] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:01.560380 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32141305-fb13-474b-bf0d-6c40cf4df1b7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.570450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32141305-fb13-474b-bf0d-6c40cf4df1b7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.575117 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8591a1-ffcd-48ba-bdb6-844df3dca464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.575924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8591a1-ffcd-48ba-bdb6-844df3dca464 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.576205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8591a1-ffcd-48ba-bdb6-844df3dca464 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.576480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8591a1-ffcd-48ba-bdb6-844df3dca464 admin admin] PATH_INFO: `/v3/projects/8042ed2089b84312b8a5960f3f3eba08/users/c4cf42bed76b4d759a1537f14f29aad5/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.578439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8591a1-ffcd-48ba-bdb6-844df3dca464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8042ed2089b84312b8a5960f3f3eba08, user_id=c4cf42bed76b4d759a1537f14f29aad5, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.581634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8591a1-ffcd-48ba-bdb6-844df3dca464 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.591615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1269/2519] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:01 2022] PUT /identity/v3/projects/8042ed2089b84312b8a5960f3f3eba08/users/c4cf42bed76b4d759a1537f14f29aad5/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:01.602931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa587d-8826-4de8-8c96-892b2a384a8e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.603308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa587d-8826-4de8-8c96-892b2a384a8e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.603599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa587d-8826-4de8-8c96-892b2a384a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.629954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-2daa587d-8826-4de8-8c96-892b2a384a8e None None] MFA Rules not processed for user `c4cf42bed76b4d759a1537f14f29aad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:01.631189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2daa587d-8826-4de8-8c96-892b2a384a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:01.698667 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1251/2520] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:01.710887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8591a1-ffcd-48ba-bdb6-844df3dca464 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.712601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cf8591a1-ffcd-48ba-bdb6-844df3dca464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:01.768990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8591a1-ffcd-48ba-bdb6-844df3dca464 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.773260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a302f-8a07-4bba-bf5d-85338a278e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.773896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a302f-8a07-4bba-bf5d-85338a278e18 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.774178 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a302f-8a07-4bba-bf5d-85338a278e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.774454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a302f-8a07-4bba-bf5d-85338a278e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.783444 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a302f-8a07-4bba-bf5d-85338a278e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.785120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a302f-8a07-4bba-bf5d-85338a278e18 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.795055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1270/2521] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:01.860147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa587d-8826-4de8-8c96-892b2a384a8e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:01.925812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa587d-8826-4de8-8c96-892b2a384a8e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:01.931230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:01.932110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:01.932396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:01.932676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] PATH_INFO: `/v3/users/64274e77a3c748d19d1615f775afd94c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:01.933887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64274e77a3c748d19d1615f775afd94c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:01.935866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:01.990439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64274e77a3c748d19d1615f775afd94c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:01.998827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64274e77a3c748d19d1615f775afd94c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.010172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64274e77a3c748d19d1615f775afd94c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.011032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64274e77a3c748d19d1615f775afd94c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.011680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64274e77a3c748d19d1615f775afd94c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.033853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:02.040253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1252/2522] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:01:01 2022] DELETE /identity/v3/users/64274e77a3c748d19d1615f775afd94c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:02.044852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02acb7-c106-4817-a68a-0b1313096d2f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:02.045191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02acb7-c106-4817-a68a-0b1313096d2f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:02.045423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02acb7-c106-4817-a68a-0b1313096d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:02.063369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6c02acb7-c106-4817-a68a-0b1313096d2f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:02.064566 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6c02acb7-c106-4817-a68a-0b1313096d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:02.123358 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1271/2523] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:02.132989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:02.134366 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:02.192025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd887e8-6e7a-4ad3-9d03-f20df97c1d13 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:02.195910 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b1ff8-ea85-439b-b738-9e118ceec86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:02.196560 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b1ff8-ea85-439b-b738-9e118ceec86f service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:02.196849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b1ff8-ea85-439b-b738-9e118ceec86f service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:02.197150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b1ff8-ea85-439b-b738-9e118ceec86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:02.206257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b1ff8-ea85-439b-b738-9e118ceec86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:02.207707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b1ff8-ea85-439b-b738-9e118ceec86f service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:02.217175 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1253/2524] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:02.486707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02acb7-c106-4817-a68a-0b1313096d2f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:02.488159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6c02acb7-c106-4817-a68a-0b1313096d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:02.501956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02acb7-c106-4817-a68a-0b1313096d2f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:02.507392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:02.508262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:02.508544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:02.508820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] PATH_INFO: `/v3/projects/4d473f097b0b47a1866a2c5ec1c4df6f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:02.510010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d473f097b0b47a1866a2c5ec1c4df6f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:02.511974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:02.557024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:02.560515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d473f097b0b47a1866a2c5ec1c4df6f', 'request_id': 'req-92374848-c13c-4923-be32-f506e6820898'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.568121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d473f097b0b47a1866a2c5ec1c4df6f', 'request_id': 'req-92374848-c13c-4923-be32-f506e6820898'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.575268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d473f097b0b47a1866a2c5ec1c4df6f', 'request_id': 'req-92374848-c13c-4923-be32-f506e6820898'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.586716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] The token cache is being invalidate because project 4d473f097b0b47a1866a2c5ec1c4df6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:01:02.587031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.587657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.589366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1272/2525] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:01:02 2022] DELETE /identity/v3/projects/4d473f097b0b47a1866a2c5ec1c4df6f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:01:02.593402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b1ff8-ea85-439b-b738-9e118ceec86f service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:02.603705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b1ff8-ea85-439b-b738-9e118ceec86f service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:02.659349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:02.660508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:02.660790 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:02.661095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] PATH_INFO: `/v3/users/c4cf42bed76b4d759a1537f14f29aad5` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:02.662319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4cf42bed76b4d759a1537f14f29aad5)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:02.664824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:02.709262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4cf42bed76b4d759a1537f14f29aad5'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.717583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4cf42bed76b4d759a1537f14f29aad5'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.724639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4cf42bed76b4d759a1537f14f29aad5'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.725927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4cf42bed76b4d759a1537f14f29aad5'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.726535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-8f9e8bb4-0f79-4afd-86d9-9ce217caed23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4cf42bed76b4d759a1537f14f29aad5'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:02.746383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1254/2526] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:01:02 2022] DELETE /identity/v3/users/c4cf42bed76b4d759a1537f14f29aad5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:03.018958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.020462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:03.034155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92374848-c13c-4923-be32-f506e6820898 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.100899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.101783 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.102054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.102339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] PATH_INFO: `/v3/projects/8042ed2089b84312b8a5960f3f3eba08` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.103589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8042ed2089b84312b8a5960f3f3eba08)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.105909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.156724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8042ed2089b84312b8a5960f3f3eba08', 'request_id': 'req-ed14ab2c-af9c-47b8-91a0-9032fafca806'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:03.163254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8042ed2089b84312b8a5960f3f3eba08', 'request_id': 'req-ed14ab2c-af9c-47b8-91a0-9032fafca806'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:03.170203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8042ed2089b84312b8a5960f3f3eba08', 'request_id': 'req-ed14ab2c-af9c-47b8-91a0-9032fafca806'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:03.181176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] The token cache is being invalidate because project 8042ed2089b84312b8a5960f3f3eba08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:01:03.181462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:03.182121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ed14ab2c-af9c-47b8-91a0-9032fafca806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:03.183795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1273/2527] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:01:03 2022] DELETE /identity/v3/projects/8042ed2089b84312b8a5960f3f3eba08 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:03.189734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bd574-732d-45a2-ba57-9e3c71a6f89b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.190023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bd574-732d-45a2-ba57-9e3c71a6f89b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.190300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bd574-732d-45a2-ba57-9e3c71a6f89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.208740 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0a3bd574-732d-45a2-ba57-9e3c71a6f89b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:03.209941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0a3bd574-732d-45a2-ba57-9e3c71a6f89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:03.222444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:03.270353 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1255/2528] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:03.277997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b62354-2ece-41aa-b6ee-f690cd9c1524 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.278286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b62354-2ece-41aa-b6ee-f690cd9c1524 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.278588 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b62354-2ece-41aa-b6ee-f690cd9c1524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.296806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-21b62354-2ece-41aa-b6ee-f690cd9c1524 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:03.297997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-21b62354-2ece-41aa-b6ee-f690cd9c1524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:03.306328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1274/2529] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:03.310529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bd574-732d-45a2-ba57-9e3c71a6f89b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.321067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bd574-732d-45a2-ba57-9e3c71a6f89b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.325726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb43795-1100-43ac-9454-97965b97f947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.326376 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb43795-1100-43ac-9454-97965b97f947 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.326800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb43795-1100-43ac-9454-97965b97f947 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.327220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb43795-1100-43ac-9454-97965b97f947 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.328000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb43795-1100-43ac-9454-97965b97f947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:01:03.328352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb43795-1100-43ac-9454-97965b97f947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.330068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb43795-1100-43ac-9454-97965b97f947 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.342539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1256/2530] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:01:03.346913 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b62354-2ece-41aa-b6ee-f690cd9c1524 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.356614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b62354-2ece-41aa-b6ee-f690cd9c1524 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.361486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34537335-afd2-4c02-9966-ff7453fe4116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.362148 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34537335-afd2-4c02-9966-ff7453fe4116 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.362435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34537335-afd2-4c02-9966-ff7453fe4116 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.362712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34537335-afd2-4c02-9966-ff7453fe4116 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.363427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34537335-afd2-4c02-9966-ff7453fe4116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.365456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34537335-afd2-4c02-9966-ff7453fe4116 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.391766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1275/2531] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:01:03 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:03.395852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb43795-1100-43ac-9454-97965b97f947 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.406655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb43795-1100-43ac-9454-97965b97f947 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.465626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50a5e4-7b99-413d-b178-381349729356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.466292 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50a5e4-7b99-413d-b178-381349729356 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.466587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50a5e4-7b99-413d-b178-381349729356 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.466889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50a5e4-7b99-413d-b178-381349729356 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.467577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50a5e4-7b99-413d-b178-381349729356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.469521 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50a5e4-7b99-413d-b178-381349729356 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.489265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1257/2532] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:01:03 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:03.494053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34537335-afd2-4c02-9966-ff7453fe4116 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.504327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34537335-afd2-4c02-9966-ff7453fe4116 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.508999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a5e5f-6aab-48d9-b5a2-3ee2824a9ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.509650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a5e5f-6aab-48d9-b5a2-3ee2824a9ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.509935 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a5e5f-6aab-48d9-b5a2-3ee2824a9ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.510218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a5e5f-6aab-48d9-b5a2-3ee2824a9ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.510973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a5e5f-6aab-48d9-b5a2-3ee2824a9ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.512567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a5e5f-6aab-48d9-b5a2-3ee2824a9ba6 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.522865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1276/2533] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:03.526576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50a5e4-7b99-413d-b178-381349729356 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.536714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50a5e4-7b99-413d-b178-381349729356 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.541843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30c12f-d8f2-4a30-999b-84195c9e9336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.542491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30c12f-d8f2-4a30-999b-84195c9e9336 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.542813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30c12f-d8f2-4a30-999b-84195c9e9336 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.543101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30c12f-d8f2-4a30-999b-84195c9e9336 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.543821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30c12f-d8f2-4a30-999b-84195c9e9336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.545422 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30c12f-d8f2-4a30-999b-84195c9e9336 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.555979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1258/2534] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:03.559748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a5e5f-6aab-48d9-b5a2-3ee2824a9ba6 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.569778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a5e5f-6aab-48d9-b5a2-3ee2824a9ba6 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.574497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55e15d-98ea-4f8f-adff-049d110d36e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.575401 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55e15d-98ea-4f8f-adff-049d110d36e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.575701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55e15d-98ea-4f8f-adff-049d110d36e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.575960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55e15d-98ea-4f8f-adff-049d110d36e7 admin admin] PATH_INFO: `/v3/projects/e9e9564032c643d58a4b3de5de983cf8/users/904925726e6e4ed0b3a0ce65b681940c/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.590350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55e15d-98ea-4f8f-adff-049d110d36e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e9564032c643d58a4b3de5de983cf8, user_id=904925726e6e4ed0b3a0ce65b681940c, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.593134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55e15d-98ea-4f8f-adff-049d110d36e7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.602088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:03.607622 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1277/2535] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:03 2022] PUT /identity/v3/projects/e9e9564032c643d58a4b3de5de983cf8/users/904925726e6e4ed0b3a0ce65b681940c/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:03.618463 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae53a6-4aab-4b7e-8762-9cf522ae9f26 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.618973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae53a6-4aab-4b7e-8762-9cf522ae9f26 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.619071 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae53a6-4aab-4b7e-8762-9cf522ae9f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.644478 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-04ae53a6-4aab-4b7e-8762-9cf522ae9f26 None None] MFA Rules not processed for user `904925726e6e4ed0b3a0ce65b681940c`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:03.645693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-04ae53a6-4aab-4b7e-8762-9cf522ae9f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:03.710484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1259/2536] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:03.714984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55e15d-98ea-4f8f-adff-049d110d36e7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.777763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55e15d-98ea-4f8f-adff-049d110d36e7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.782446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd6c4f-c2fc-4298-ab9f-7a3b1dbcd46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.783127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd6c4f-c2fc-4298-ab9f-7a3b1dbcd46c admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.783429 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd6c4f-c2fc-4298-ab9f-7a3b1dbcd46c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.783722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd6c4f-c2fc-4298-ab9f-7a3b1dbcd46c admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.784441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd6c4f-c2fc-4298-ab9f-7a3b1dbcd46c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.786611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd6c4f-c2fc-4298-ab9f-7a3b1dbcd46c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.812440 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1278/2537] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:01:03 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:03.816609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae53a6-4aab-4b7e-8762-9cf522ae9f26 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.827171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae53a6-4aab-4b7e-8762-9cf522ae9f26 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.838795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:03.884083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e1345-3d6c-4234-96d3-207ae09a1cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.884752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e1345-3d6c-4234-96d3-207ae09a1cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.885053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e1345-3d6c-4234-96d3-207ae09a1cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.885332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e1345-3d6c-4234-96d3-207ae09a1cf9 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.886013 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e1345-3d6c-4234-96d3-207ae09a1cf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.887956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e1345-3d6c-4234-96d3-207ae09a1cf9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.907986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1260/2538] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:01:03 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:03.912272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd6c4f-c2fc-4298-ab9f-7a3b1dbcd46c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.922962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd6c4f-c2fc-4298-ab9f-7a3b1dbcd46c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.927452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6e2e-3964-461e-9b80-cfa254a39521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.928097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6e2e-3964-461e-9b80-cfa254a39521 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.928373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6e2e-3964-461e-9b80-cfa254a39521 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.928644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6e2e-3964-461e-9b80-cfa254a39521 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.929351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a6e2e-3964-461e-9b80-cfa254a39521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.931013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a6e2e-3964-461e-9b80-cfa254a39521 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.941247 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1279/2539] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:01:03.945392 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e1345-3d6c-4234-96d3-207ae09a1cf9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:03.955299 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e1345-3d6c-4234-96d3-207ae09a1cf9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:03.960400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e32b88-6cf9-471c-a076-1655557b64a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:03.961270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e32b88-6cf9-471c-a076-1655557b64a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:03.961556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e32b88-6cf9-471c-a076-1655557b64a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:03.961831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e32b88-6cf9-471c-a076-1655557b64a5 admin admin] PATH_INFO: `/v3/projects/5fe639b5e74d46be9fb1502284862575/users/a6b130d0c2144add85b05cce2715aa4b/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:03.977230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e32b88-6cf9-471c-a076-1655557b64a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe639b5e74d46be9fb1502284862575, user_id=a6b130d0c2144add85b05cce2715aa4b, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:03.979880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e32b88-6cf9-471c-a076-1655557b64a5 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:03.992428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1261/2540] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:03 2022] PUT /identity/v3/projects/5fe639b5e74d46be9fb1502284862575/users/a6b130d0c2144add85b05cce2715aa4b/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:03.996428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6e2e-3964-461e-9b80-cfa254a39521 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:04.006393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6e2e-3964-461e-9b80-cfa254a39521 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:04.063875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15395b1b-4b34-4bdb-be9d-f4e4ff13d2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:04.064535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15395b1b-4b34-4bdb-be9d-f4e4ff13d2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:04.064816 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15395b1b-4b34-4bdb-be9d-f4e4ff13d2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:04.065091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15395b1b-4b34-4bdb-be9d-f4e4ff13d2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:04.065802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15395b1b-4b34-4bdb-be9d-f4e4ff13d2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:04.067439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15395b1b-4b34-4bdb-be9d-f4e4ff13d2d8 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:04.077007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1280/2541] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:04.081053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e32b88-6cf9-471c-a076-1655557b64a5 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:04.091645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e32b88-6cf9-471c-a076-1655557b64a5 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:04.096419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07980486-8236-4862-a5e8-85f9bff88faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:04.097289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980486-8236-4862-a5e8-85f9bff88faf admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:04.097573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980486-8236-4862-a5e8-85f9bff88faf admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:04.097846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980486-8236-4862-a5e8-85f9bff88faf admin admin] PATH_INFO: `/v3/projects/5fe639b5e74d46be9fb1502284862575/users/a6b130d0c2144add85b05cce2715aa4b/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:04.099824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07980486-8236-4862-a5e8-85f9bff88faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe639b5e74d46be9fb1502284862575, user_id=a6b130d0c2144add85b05cce2715aa4b, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:04.103057 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07980486-8236-4862-a5e8-85f9bff88faf admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:04.113130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1262/2542] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:04 2022] PUT /identity/v3/projects/5fe639b5e74d46be9fb1502284862575/users/a6b130d0c2144add85b05cce2715aa4b/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:04.123928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a45b97-b666-4322-a588-1e2a9c628da5 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:04.124292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a45b97-b666-4322-a588-1e2a9c628da5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:04.124575 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a45b97-b666-4322-a588-1e2a9c628da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:04.150607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-74a45b97-b666-4322-a588-1e2a9c628da5 None None] MFA Rules not processed for user `a6b130d0c2144add85b05cce2715aa4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:04.151775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-74a45b97-b666-4322-a588-1e2a9c628da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:04.219874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:04.230956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1281/2543] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:04.239753 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07980486-8236-4862-a5e8-85f9bff88faf admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:04.241624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-07980486-8236-4862-a5e8-85f9bff88faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:04.306368 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07980486-8236-4862-a5e8-85f9bff88faf admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:04.310581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fd3a1-5f79-470d-9ef5-75a1c40e80ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:04.311269 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fd3a1-5f79-470d-9ef5-75a1c40e80ed service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:04.311552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fd3a1-5f79-470d-9ef5-75a1c40e80ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:04.311834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fd3a1-5f79-470d-9ef5-75a1c40e80ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:04.374641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fd3a1-5f79-470d-9ef5-75a1c40e80ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:04.376142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fd3a1-5f79-470d-9ef5-75a1c40e80ed service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:04.385589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1263/2544] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:06.919392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a45b97-b666-4322-a588-1e2a9c628da5 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:06.981028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a45b97-b666-4322-a588-1e2a9c628da5 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:06.985554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:06.986419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:06.986701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:06.987017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] PATH_INFO: `/v3/users/904925726e6e4ed0b3a0ce65b681940c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:06.988194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=904925726e6e4ed0b3a0ce65b681940c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:06.990153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:07.036526 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '904925726e6e4ed0b3a0ce65b681940c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.043949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '904925726e6e4ed0b3a0ce65b681940c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.051468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '904925726e6e4ed0b3a0ce65b681940c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.052439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '904925726e6e4ed0b3a0ce65b681940c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.065285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '904925726e6e4ed0b3a0ce65b681940c'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.071820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1282/2545] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:01:06 2022] DELETE /identity/v3/users/904925726e6e4ed0b3a0ce65b681940c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:07.076313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece0bbd-fafe-4211-9560-f4992895f5e0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:07.076786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece0bbd-fafe-4211-9560-f4992895f5e0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:07.076899 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece0bbd-fafe-4211-9560-f4992895f5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:07.094257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-3ece0bbd-fafe-4211-9560-f4992895f5e0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:07.095484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3ece0bbd-fafe-4211-9560-f4992895f5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:07.111599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:07.159361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1264/2546] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:07.169377 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:07.170877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:07.230026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101acf0d-68c9-49ce-b727-0f39cbe4afdf admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:07.233742 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef9bf-a6ae-4279-bd4c-f1a3ef698c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:07.234399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef9bf-a6ae-4279-bd4c-f1a3ef698c79 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:07.234676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef9bf-a6ae-4279-bd4c-f1a3ef698c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:07.235013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef9bf-a6ae-4279-bd4c-f1a3ef698c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:07.245204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef9bf-a6ae-4279-bd4c-f1a3ef698c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:07.246627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef9bf-a6ae-4279-bd4c-f1a3ef698c79 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:07.255910 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1283/2547] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:07.530288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece0bbd-fafe-4211-9560-f4992895f5e0 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:07.531802 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3ece0bbd-fafe-4211-9560-f4992895f5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:07.547946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece0bbd-fafe-4211-9560-f4992895f5e0 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:07.552838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:07.553711 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:07.554593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:07.555020 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] PATH_INFO: `/v3/projects/e9e9564032c643d58a4b3de5de983cf8` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:07.556210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e9564032c643d58a4b3de5de983cf8)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:07.558272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:07.607398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e9564032c643d58a4b3de5de983cf8', 'request_id': 'req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.613624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e9564032c643d58a4b3de5de983cf8', 'request_id': 'req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.620991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e9564032c643d58a4b3de5de983cf8', 'request_id': 'req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.631934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] The token cache is being invalidate because project e9e9564032c643d58a4b3de5de983cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:01:07.632266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.632968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.634516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1265/2548] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:01:07 2022] DELETE /identity/v3/projects/e9e9564032c643d58a4b3de5de983cf8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:07.638395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef9bf-a6ae-4279-bd4c-f1a3ef698c79 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:07.648712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef9bf-a6ae-4279-bd4c-f1a3ef698c79 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:07.661823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:07.707126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:07.707995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:07.708273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:07.708548 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] PATH_INFO: `/v3/users/a6b130d0c2144add85b05cce2715aa4b` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:07.710066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6b130d0c2144add85b05cce2715aa4b)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:07.712209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:07.765058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6b130d0c2144add85b05cce2715aa4b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.772496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6b130d0c2144add85b05cce2715aa4b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.779944 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6b130d0c2144add85b05cce2715aa4b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.780559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6b130d0c2144add85b05cce2715aa4b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.793414 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c9207ced-e911-4784-a1d2-fd7bdb7322a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6b130d0c2144add85b05cce2715aa4b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:07.799478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1284/2549] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:01:07 2022] DELETE /identity/v3/users/a6b130d0c2144add85b05cce2715aa4b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:08.088625 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:08.090210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:08.105195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b2974-8efe-48b4-bab2-a2bb4fedbc07 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:08.166566 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:08.167544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.167826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.168101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] PATH_INFO: `/v3/projects/5fe639b5e74d46be9fb1502284862575` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.169278 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe639b5e74d46be9fb1502284862575)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:08.171390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:08.218168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:08.219382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe639b5e74d46be9fb1502284862575', 'request_id': 'req-1908aaca-7612-48d4-ab64-61fe580cd5b4'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:08.227363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe639b5e74d46be9fb1502284862575', 'request_id': 'req-1908aaca-7612-48d4-ab64-61fe580cd5b4'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:08.234942 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe639b5e74d46be9fb1502284862575', 'request_id': 'req-1908aaca-7612-48d4-ab64-61fe580cd5b4'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:08.248277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] The token cache is being invalidate because project 5fe639b5e74d46be9fb1502284862575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:01:08.248579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:08.249222 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-1908aaca-7612-48d4-ab64-61fe580cd5b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:08.250787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1266/2550] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:01:08 2022] DELETE /identity/v3/projects/5fe639b5e74d46be9fb1502284862575 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:08.256030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baafea4-44e1-4ad9-9bb2-34312d6543d5 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.256515 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baafea4-44e1-4ad9-9bb2-34312d6543d5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.256624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baafea4-44e1-4ad9-9bb2-34312d6543d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.276153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9baafea4-44e1-4ad9-9bb2-34312d6543d5 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:08.277405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9baafea4-44e1-4ad9-9bb2-34312d6543d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:08.342030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1285/2551] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:08.349528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c6aef-51fc-47a1-b96e-65e4dd0e46ac None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.349821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c6aef-51fc-47a1-b96e-65e4dd0e46ac None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.350104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c6aef-51fc-47a1-b96e-65e4dd0e46ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.370181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-eb3c6aef-51fc-47a1-b96e-65e4dd0e46ac None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:08.371763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-eb3c6aef-51fc-47a1-b96e-65e4dd0e46ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:08.381120 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1267/2552] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:08.386535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baafea4-44e1-4ad9-9bb2-34312d6543d5 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:08.396512 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baafea4-44e1-4ad9-9bb2-34312d6543d5 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:08.401128 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21ffa7-d3ec-4be3-8cb7-cbc33a3cc569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:08.401812 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21ffa7-d3ec-4be3-8cb7-cbc33a3cc569 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.402099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21ffa7-d3ec-4be3-8cb7-cbc33a3cc569 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.402379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21ffa7-d3ec-4be3-8cb7-cbc33a3cc569 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.403144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21ffa7-d3ec-4be3-8cb7-cbc33a3cc569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:01:08.403458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21ffa7-d3ec-4be3-8cb7-cbc33a3cc569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:08.405060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21ffa7-d3ec-4be3-8cb7-cbc33a3cc569 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:08.417490 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1286/2553] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 16:01:08.421452 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c6aef-51fc-47a1-b96e-65e4dd0e46ac None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:08.431430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c6aef-51fc-47a1-b96e-65e4dd0e46ac None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:08.436218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d2469-d533-4c6f-8012-e9754c3be7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:08.436870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d2469-d533-4c6f-8012-e9754c3be7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.437153 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d2469-d533-4c6f-8012-e9754c3be7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.437431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d2469-d533-4c6f-8012-e9754c3be7e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.438115 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d2469-d533-4c6f-8012-e9754c3be7e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:08.440051 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d2469-d533-4c6f-8012-e9754c3be7e6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:08.466638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1268/2554] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:01:08 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:08.471102 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21ffa7-d3ec-4be3-8cb7-cbc33a3cc569 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:08.481503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21ffa7-d3ec-4be3-8cb7-cbc33a3cc569 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:08.502657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:08.541594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c91ee2-6aab-4a6a-bb9e-6abe08be407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:08.542260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c91ee2-6aab-4a6a-bb9e-6abe08be407b admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.542542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c91ee2-6aab-4a6a-bb9e-6abe08be407b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.542852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c91ee2-6aab-4a6a-bb9e-6abe08be407b admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.543535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c91ee2-6aab-4a6a-bb9e-6abe08be407b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:08.545455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c91ee2-6aab-4a6a-bb9e-6abe08be407b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:08.565694 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1287/2555] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:01:08 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:01:08.570314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d2469-d533-4c6f-8012-e9754c3be7e6 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:08.580499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d2469-d533-4c6f-8012-e9754c3be7e6 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:08.585064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5953535-95fa-4bb6-8301-1458a0ea4a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:08.585739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5953535-95fa-4bb6-8301-1458a0ea4a2e admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.586021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5953535-95fa-4bb6-8301-1458a0ea4a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.586297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5953535-95fa-4bb6-8301-1458a0ea4a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.587063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5953535-95fa-4bb6-8301-1458a0ea4a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:08.588684 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5953535-95fa-4bb6-8301-1458a0ea4a2e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:08.599219 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1269/2556] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:08.603215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c91ee2-6aab-4a6a-bb9e-6abe08be407b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:08.613317 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c91ee2-6aab-4a6a-bb9e-6abe08be407b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:08.619275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae052fe-f085-4256-9b3d-e2e0d25d54b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:08.619939 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae052fe-f085-4256-9b3d-e2e0d25d54b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.620222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae052fe-f085-4256-9b3d-e2e0d25d54b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.620503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae052fe-f085-4256-9b3d-e2e0d25d54b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.621237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae052fe-f085-4256-9b3d-e2e0d25d54b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:08.622912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae052fe-f085-4256-9b3d-e2e0d25d54b1 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:08.633956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1288/2557] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:08.638058 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5953535-95fa-4bb6-8301-1458a0ea4a2e admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:08.647933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5953535-95fa-4bb6-8301-1458a0ea4a2e admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:08.652626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc622e-1668-4a98-b2e3-80692d1e5392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:08.653772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc622e-1668-4a98-b2e3-80692d1e5392 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.654065 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc622e-1668-4a98-b2e3-80692d1e5392 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.654336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc622e-1668-4a98-b2e3-80692d1e5392 admin admin] PATH_INFO: `/v3/projects/76e5ca6426484a9f8b025cfe5241c393/users/4a98f10a310347fe8fd9863b7142deba/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.669456 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc622e-1668-4a98-b2e3-80692d1e5392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e5ca6426484a9f8b025cfe5241c393, user_id=4a98f10a310347fe8fd9863b7142deba, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:08.672108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc622e-1668-4a98-b2e3-80692d1e5392 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:08.685482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1270/2558] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:08 2022] PUT /identity/v3/projects/76e5ca6426484a9f8b025cfe5241c393/users/4a98f10a310347fe8fd9863b7142deba/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:08.697024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520f3ea-9fe0-4c10-8c83-a01e9f692536 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.697315 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520f3ea-9fe0-4c10-8c83-a01e9f692536 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.697598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520f3ea-9fe0-4c10-8c83-a01e9f692536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.725013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-0520f3ea-9fe0-4c10-8c83-a01e9f692536 None None] MFA Rules not processed for user `4a98f10a310347fe8fd9863b7142deba`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:08.726625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0520f3ea-9fe0-4c10-8c83-a01e9f692536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:08.796446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1289/2559] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:01:08.807025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc622e-1668-4a98-b2e3-80692d1e5392 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:08.808779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-13fc622e-1668-4a98-b2e3-80692d1e5392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:08.865941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc622e-1668-4a98-b2e3-80692d1e5392 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:08.870244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83f634-adb0-4c5c-a291-a700f2cdef16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:08.870928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83f634-adb0-4c5c-a291-a700f2cdef16 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:08.871214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83f634-adb0-4c5c-a291-a700f2cdef16 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:08.871520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83f634-adb0-4c5c-a291-a700f2cdef16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:08.879905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83f634-adb0-4c5c-a291-a700f2cdef16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:08.881328 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83f634-adb0-4c5c-a291-a700f2cdef16 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:08.890710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1271/2560] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:11.915086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520f3ea-9fe0-4c10-8c83-a01e9f692536 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:11.981385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520f3ea-9fe0-4c10-8c83-a01e9f692536 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:11.985835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:11.986707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:11.987017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:11.987292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] PATH_INFO: `/v3/users/4a98f10a310347fe8fd9863b7142deba` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:11.988470 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a98f10a310347fe8fd9863b7142deba)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:11.990384 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:12.028676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:12.040616 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a98f10a310347fe8fd9863b7142deba'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.047499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a98f10a310347fe8fd9863b7142deba'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.055499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a98f10a310347fe8fd9863b7142deba'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.056127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a98f10a310347fe8fd9863b7142deba'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.068559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a98f10a310347fe8fd9863b7142deba'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.074449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1290/2561] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:01:11 2022] DELETE /identity/v3/users/4a98f10a310347fe8fd9863b7142deba => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:12.078967 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf81d1-8e88-4950-8f4a-35ac1bedcd2e None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:12.079444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf81d1-8e88-4950-8f4a-35ac1bedcd2e None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:12.079564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf81d1-8e88-4950-8f4a-35ac1bedcd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:12.100860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0acf81d1-8e88-4950-8f4a-35ac1bedcd2e None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:12.101968 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0acf81d1-8e88-4950-8f4a-35ac1bedcd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:12.163256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1272/2562] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:12.174188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:12.175670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:12.233993 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac03836-5c3c-4c75-8851-a55c71e1e218 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:12.237653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376f77f-2635-4c3a-8320-39f4af3b9bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:12.238304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376f77f-2635-4c3a-8320-39f4af3b9bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:12.238585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376f77f-2635-4c3a-8320-39f4af3b9bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:12.238893 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376f77f-2635-4c3a-8320-39f4af3b9bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:12.248977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376f77f-2635-4c3a-8320-39f4af3b9bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:12.250450 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376f77f-2635-4c3a-8320-39f4af3b9bd0 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:12.260779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1291/2563] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:12.533434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf81d1-8e88-4950-8f4a-35ac1bedcd2e None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:12.534970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0acf81d1-8e88-4950-8f4a-35ac1bedcd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:12.547689 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf81d1-8e88-4950-8f4a-35ac1bedcd2e None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:12.555889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:12.556645 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:12.556928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:12.557206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] PATH_INFO: `/v3/projects/76e5ca6426484a9f8b025cfe5241c393` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:12.558424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e5ca6426484a9f8b025cfe5241c393)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:12.560449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:12.571934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:12.610630 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e5ca6426484a9f8b025cfe5241c393', 'request_id': 'req-e7123819-b80f-4d83-8d2e-b2e6ebed469d'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.617787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e5ca6426484a9f8b025cfe5241c393', 'request_id': 'req-e7123819-b80f-4d83-8d2e-b2e6ebed469d'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.626144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e5ca6426484a9f8b025cfe5241c393', 'request_id': 'req-e7123819-b80f-4d83-8d2e-b2e6ebed469d'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.636605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] The token cache is being invalidate because project 76e5ca6426484a9f8b025cfe5241c393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:01:12.636900 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.637541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e7123819-b80f-4d83-8d2e-b2e6ebed469d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:12.639171 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1273/2564] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:01:12 2022] DELETE /identity/v3/projects/76e5ca6426484a9f8b025cfe5241c393 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:12.644164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84df5-90d8-440d-b5c4-71f77f65bf83 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:12.644457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84df5-90d8-440d-b5c4-71f77f65bf83 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:12.644740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84df5-90d8-440d-b5c4-71f77f65bf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:12.662425 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-1cb84df5-90d8-440d-b5c4-71f77f65bf83 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:12.664158 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1cb84df5-90d8-440d-b5c4-71f77f65bf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:12.723425 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1292/2565] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:12.732399 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58168270-2623-4ef1-885f-58f663b71e37 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:12.732692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58168270-2623-4ef1-885f-58f663b71e37 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:12.733017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58168270-2623-4ef1-885f-58f663b71e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:12.751759 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-58168270-2623-4ef1-885f-58f663b71e37 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:12.752972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-58168270-2623-4ef1-885f-58f663b71e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:12.761041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1274/2566] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:12.765242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb84df5-90d8-440d-b5c4-71f77f65bf83 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:12.775047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb84df5-90d8-440d-b5c4-71f77f65bf83 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:12.779635 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4e519-1da5-4aa7-abb5-345b4f6fee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:12.780283 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4e519-1da5-4aa7-abb5-345b4f6fee34 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:12.780599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4e519-1da5-4aa7-abb5-345b4f6fee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:12.780878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4e519-1da5-4aa7-abb5-345b4f6fee34 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:12.781598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4e519-1da5-4aa7-abb5-345b4f6fee34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:01:12.781907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4e519-1da5-4aa7-abb5-345b4f6fee34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:12.783543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4e519-1da5-4aa7-abb5-345b4f6fee34 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:12.799206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1293/2567] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:01:12.800461 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58168270-2623-4ef1-885f-58f663b71e37 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:12.810352 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58168270-2623-4ef1-885f-58f663b71e37 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:12.815203 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd878f6-3891-41ed-b2f4-2043da87f6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:12.815819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd878f6-3891-41ed-b2f4-2043da87f6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:12.816056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd878f6-3891-41ed-b2f4-2043da87f6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:12.816336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd878f6-3891-41ed-b2f4-2043da87f6f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:12.817022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd878f6-3891-41ed-b2f4-2043da87f6f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:12.818992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd878f6-3891-41ed-b2f4-2043da87f6f7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:12.847831 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1275/2568] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:01:12 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:12.850524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4e519-1da5-4aa7-abb5-345b4f6fee34 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:12.861326 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4e519-1da5-4aa7-abb5-345b4f6fee34 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:12.901618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:12.920703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6fc96-96fa-4afa-829b-454a8ad8dc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:12.921376 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6fc96-96fa-4afa-829b-454a8ad8dc39 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:12.921655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6fc96-96fa-4afa-829b-454a8ad8dc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:12.921933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6fc96-96fa-4afa-829b-454a8ad8dc39 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:12.922617 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6fc96-96fa-4afa-829b-454a8ad8dc39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:12.924590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6fc96-96fa-4afa-829b-454a8ad8dc39 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:12.944826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1294/2569] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:01:12 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:12.948942 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd878f6-3891-41ed-b2f4-2043da87f6f7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:12.959158 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd878f6-3891-41ed-b2f4-2043da87f6f7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:12.965329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5773fa0-687a-46d3-992d-53573168e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:12.966093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5773fa0-687a-46d3-992d-53573168e2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:12.966429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5773fa0-687a-46d3-992d-53573168e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:12.966708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5773fa0-687a-46d3-992d-53573168e2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:12.967465 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5773fa0-687a-46d3-992d-53573168e2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:12.969089 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5773fa0-687a-46d3-992d-53573168e2b2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:12.981872 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1276/2570] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:12.985877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6fc96-96fa-4afa-829b-454a8ad8dc39 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:13.003248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6fc96-96fa-4afa-829b-454a8ad8dc39 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:13.008228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f38fd-593f-41af-97a5-342215d0b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:13.008924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f38fd-593f-41af-97a5-342215d0b916 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:13.009210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f38fd-593f-41af-97a5-342215d0b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:13.009486 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f38fd-593f-41af-97a5-342215d0b916 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:13.010222 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f38fd-593f-41af-97a5-342215d0b916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:13.011858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f38fd-593f-41af-97a5-342215d0b916 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:13.022422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1295/2571] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:13.027003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5773fa0-687a-46d3-992d-53573168e2b2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:13.037533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5773fa0-687a-46d3-992d-53573168e2b2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:13.042791 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b1d2-6cb2-4440-a170-07ed8ee295dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:13.043675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b1d2-6cb2-4440-a170-07ed8ee295dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:13.043956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b1d2-6cb2-4440-a170-07ed8ee295dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:13.044230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b1d2-6cb2-4440-a170-07ed8ee295dd admin admin] PATH_INFO: `/v3/projects/31480d73f44447679761a5acac79dc02/users/f18270589e8745a286e3c78a7f86b25d/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:13.058660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005b1d2-6cb2-4440-a170-07ed8ee295dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=31480d73f44447679761a5acac79dc02, user_id=f18270589e8745a286e3c78a7f86b25d, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:13.061330 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005b1d2-6cb2-4440-a170-07ed8ee295dd admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:13.074313 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1277/2572] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:13 2022] PUT /identity/v3/projects/31480d73f44447679761a5acac79dc02/users/f18270589e8745a286e3c78a7f86b25d/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:13.079391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e8bc1-19f5-4d21-84a5-3eb76812d2fd None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:13.079677 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e8bc1-19f5-4d21-84a5-3eb76812d2fd None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:13.079955 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e8bc1-19f5-4d21-84a5-3eb76812d2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:13.099746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-863e8bc1-19f5-4d21-84a5-3eb76812d2fd None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:13.100957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-863e8bc1-19f5-4d21-84a5-3eb76812d2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:13.159639 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1296/2573] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:13.170455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b1d2-6cb2-4440-a170-07ed8ee295dd admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:13.172235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0005b1d2-6cb2-4440-a170-07ed8ee295dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:13.230155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b1d2-6cb2-4440-a170-07ed8ee295dd admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:13.233862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90284d-14d3-4ead-981c-95cef4677eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:13.234448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90284d-14d3-4ead-981c-95cef4677eba service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:13.234754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90284d-14d3-4ead-981c-95cef4677eba service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:13.235033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90284d-14d3-4ead-981c-95cef4677eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:13.244505 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90284d-14d3-4ead-981c-95cef4677eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:13.245927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90284d-14d3-4ead-981c-95cef4677eba service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:13.255325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1278/2574] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:17.195880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71378d-ec8c-4643-be74-c2183a6b3074 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:17.196391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71378d-ec8c-4643-be74-c2183a6b3074 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:17.196501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71378d-ec8c-4643-be74-c2183a6b3074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:17.227505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-0e71378d-ec8c-4643-be74-c2183a6b3074 None None] MFA Rules not processed for user `f18270589e8745a286e3c78a7f86b25d`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:17.228736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0e71378d-ec8c-4643-be74-c2183a6b3074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:17.303665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1297/2575] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:17.308218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90284d-14d3-4ead-981c-95cef4677eba service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:17.318014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90284d-14d3-4ead-981c-95cef4677eba service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:17.323259 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246d179-959d-459f-bebd-735570d3a5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:17.323926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246d179-959d-459f-bebd-735570d3a5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:17.324211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246d179-959d-459f-bebd-735570d3a5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:17.324491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246d179-959d-459f-bebd-735570d3a5b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:17.325195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246d179-959d-459f-bebd-735570d3a5b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:17.327147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246d179-959d-459f-bebd-735570d3a5b9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:17.376236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1279/2576] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:01:17 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:17.380442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71378d-ec8c-4643-be74-c2183a6b3074 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:17.399757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71378d-ec8c-4643-be74-c2183a6b3074 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:17.455111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:17.467473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8f4bc-6fb9-468a-a862-8c3ce491fe74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:17.468266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8f4bc-6fb9-468a-a862-8c3ce491fe74 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:17.468616 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8f4bc-6fb9-468a-a862-8c3ce491fe74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:17.469024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8f4bc-6fb9-468a-a862-8c3ce491fe74 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:17.469820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8f4bc-6fb9-468a-a862-8c3ce491fe74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:17.471892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8f4bc-6fb9-468a-a862-8c3ce491fe74 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:17.493948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1298/2577] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:01:17 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:17.497875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246d179-959d-459f-bebd-735570d3a5b9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:17.508991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246d179-959d-459f-bebd-735570d3a5b9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:17.513639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0e803-d3e6-46d8-bfeb-8d97442f3605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:17.514308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0e803-d3e6-46d8-bfeb-8d97442f3605 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:17.514592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0e803-d3e6-46d8-bfeb-8d97442f3605 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:17.514906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0e803-d3e6-46d8-bfeb-8d97442f3605 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:17.515643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0e803-d3e6-46d8-bfeb-8d97442f3605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:17.517274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0e803-d3e6-46d8-bfeb-8d97442f3605 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:17.528817 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1280/2578] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:17.532758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8f4bc-6fb9-468a-a862-8c3ce491fe74 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:17.543505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8f4bc-6fb9-468a-a862-8c3ce491fe74 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:17.548336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9733f90-3922-4812-bc80-11136cfe3a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:17.549262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9733f90-3922-4812-bc80-11136cfe3a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:17.549554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9733f90-3922-4812-bc80-11136cfe3a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:17.549863 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9733f90-3922-4812-bc80-11136cfe3a2f admin admin] PATH_INFO: `/v3/projects/2cab9b0de3b445559fa85e7a5a2cd03b/users/ac3a4820a4b442e0b3b3f6cc36053cf2/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:17.566995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9733f90-3922-4812-bc80-11136cfe3a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cab9b0de3b445559fa85e7a5a2cd03b, user_id=ac3a4820a4b442e0b3b3f6cc36053cf2, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:17.569673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9733f90-3922-4812-bc80-11136cfe3a2f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:17.583899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1299/2579] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:17 2022] PUT /identity/v3/projects/2cab9b0de3b445559fa85e7a5a2cd03b/users/ac3a4820a4b442e0b3b3f6cc36053cf2/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:17.587948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0e803-d3e6-46d8-bfeb-8d97442f3605 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:17.594588 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:17.599475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0e803-d3e6-46d8-bfeb-8d97442f3605 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:17.668235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0905c-6adc-4469-94d9-26776031e80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:17.668835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0905c-6adc-4469-94d9-26776031e80a admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:17.669121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0905c-6adc-4469-94d9-26776031e80a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:17.669400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0905c-6adc-4469-94d9-26776031e80a admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:17.670285 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0905c-6adc-4469-94d9-26776031e80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:17.672108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0905c-6adc-4469-94d9-26776031e80a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:17.683591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1281/2580] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:17.687787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9733f90-3922-4812-bc80-11136cfe3a2f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:17.699260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9733f90-3922-4812-bc80-11136cfe3a2f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:17.704099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136f17f-ca38-4760-8fda-64a3f1926ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:17.705043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136f17f-ca38-4760-8fda-64a3f1926ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:17.705328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136f17f-ca38-4760-8fda-64a3f1926ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:17.705605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136f17f-ca38-4760-8fda-64a3f1926ce2 admin admin] PATH_INFO: `/v3/projects/2cab9b0de3b445559fa85e7a5a2cd03b/users/ac3a4820a4b442e0b3b3f6cc36053cf2/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:17.707643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136f17f-ca38-4760-8fda-64a3f1926ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cab9b0de3b445559fa85e7a5a2cd03b, user_id=ac3a4820a4b442e0b3b3f6cc36053cf2, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:17.710327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136f17f-ca38-4760-8fda-64a3f1926ce2 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:17.722058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1300/2581] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:17 2022] PUT /identity/v3/projects/2cab9b0de3b445559fa85e7a5a2cd03b/users/ac3a4820a4b442e0b3b3f6cc36053cf2/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:22.047363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2295939-5b91-4fec-bab9-42de6f8717b3 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:22.047889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2295939-5b91-4fec-bab9-42de6f8717b3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:22.048015 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2295939-5b91-4fec-bab9-42de6f8717b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:22.074288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e2295939-5b91-4fec-bab9-42de6f8717b3 None None] MFA Rules not processed for user `ac3a4820a4b442e0b3b3f6cc36053cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:22.075485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e2295939-5b91-4fec-bab9-42de6f8717b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:22.152253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1282/2582] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:22.167442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136f17f-ca38-4760-8fda-64a3f1926ce2 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:22.169583 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0136f17f-ca38-4760-8fda-64a3f1926ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:22.263193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136f17f-ca38-4760-8fda-64a3f1926ce2 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:22.269671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b9ba4-7402-40a4-9480-807b3d228029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:22.270333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b9ba4-7402-40a4-9480-807b3d228029 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:22.270618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b9ba4-7402-40a4-9480-807b3d228029 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:22.270980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b9ba4-7402-40a4-9480-807b3d228029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:22.331883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b9ba4-7402-40a4-9480-807b3d228029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:22.333921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b9ba4-7402-40a4-9480-807b3d228029 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:22.345180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1301/2583] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:29.402423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2295939-5b91-4fec-bab9-42de6f8717b3 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:29.405676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e2295939-5b91-4fec-bab9-42de6f8717b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:29.422752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2295939-5b91-4fec-bab9-42de6f8717b3 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:29.428458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f5411-4c52-4b04-b3d9-2e26460556ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:29.429108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5411-4c52-4b04-b3d9-2e26460556ea service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:29.429398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5411-4c52-4b04-b3d9-2e26460556ea service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:29.429681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5411-4c52-4b04-b3d9-2e26460556ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:29.431630 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8a1f5411-4c52-4b04-b3d9-2e26460556ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:29.511307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f5411-4c52-4b04-b3d9-2e26460556ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:29.512880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f5411-4c52-4b04-b3d9-2e26460556ea service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:29.522950 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:29.532273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1283/2584] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:32.340781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:34.919835 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b9ba4-7402-40a4-9480-807b3d228029 service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:34.921336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-386b9ba4-7402-40a4-9480-807b3d228029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:34.988765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b9ba4-7402-40a4-9480-807b3d228029 service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:34.994037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:34.994948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:34.995234 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:34.995513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] PATH_INFO: `/v3/users/4582a193951844fb902a088ee77ac112` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:34.996827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4582a193951844fb902a088ee77ac112)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:34.998817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:35.044724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4582a193951844fb902a088ee77ac112'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.052204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4582a193951844fb902a088ee77ac112'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.060878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4582a193951844fb902a088ee77ac112'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.061618 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4582a193951844fb902a088ee77ac112'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.073276 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4582a193951844fb902a088ee77ac112'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.079215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1302/2585] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:01:34 2022] DELETE /identity/v3/users/4582a193951844fb902a088ee77ac112 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:35.083681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3b00b-4523-42ce-b593-c031fec5b944 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:35.084168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3b00b-4523-42ce-b593-c031fec5b944 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:35.084280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3b00b-4523-42ce-b593-c031fec5b944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:35.102354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f3f3b00b-4523-42ce-b593-c031fec5b944 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:35.103577 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f3f3b00b-4523-42ce-b593-c031fec5b944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:35.162693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1284/2586] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:35.172795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:35.174209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:35.231885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c2c37-6ed5-41f1-8cc7-6a5a3c59c10e admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:35.235787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83692d-d554-47ca-8abf-659090765f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:35.236433 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83692d-d554-47ca-8abf-659090765f8f service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:35.236712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83692d-d554-47ca-8abf-659090765f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:35.236990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83692d-d554-47ca-8abf-659090765f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:35.246506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83692d-d554-47ca-8abf-659090765f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:35.247979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83692d-d554-47ca-8abf-659090765f8f service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:35.258424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1303/2587] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:35.548524 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3b00b-4523-42ce-b593-c031fec5b944 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:35.550053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f3f3b00b-4523-42ce-b593-c031fec5b944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:35.569348 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3b00b-4523-42ce-b593-c031fec5b944 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:35.576442 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:35.577460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:35.577720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:35.578001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] PATH_INFO: `/v3/projects/d238672f4f014ad89541209711b8c0bb` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:35.579256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d238672f4f014ad89541209711b8c0bb)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:35.581495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:35.632928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd238672f4f014ad89541209711b8c0bb', 'request_id': 'req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.639100 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd238672f4f014ad89541209711b8c0bb', 'request_id': 'req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.646635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd238672f4f014ad89541209711b8c0bb', 'request_id': 'req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.658006 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] The token cache is being invalidate because project d238672f4f014ad89541209711b8c0bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:01:35.658289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.658966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-ed0af992-7c0b-45ae-b8e4-9a2eff8d2e7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:35.660520 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1285/2588] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:01:35 2022] DELETE /identity/v3/projects/d238672f4f014ad89541209711b8c0bb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:35.666817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef4b01-4259-4a2d-8446-f322a4bacd85 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:35.667060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef4b01-4259-4a2d-8446-f322a4bacd85 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:35.667339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef4b01-4259-4a2d-8446-f322a4bacd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:35.684977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-62ef4b01-4259-4a2d-8446-f322a4bacd85 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:35.686191 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-62ef4b01-4259-4a2d-8446-f322a4bacd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:35.704396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:35.749311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1304/2589] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:35.757595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aa959-58c4-4ce2-b72e-698d28c7edbd None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:35.757886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aa959-58c4-4ce2-b72e-698d28c7edbd None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:35.758166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aa959-58c4-4ce2-b72e-698d28c7edbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:35.777511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-529aa959-58c4-4ce2-b72e-698d28c7edbd None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:35.778707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-529aa959-58c4-4ce2-b72e-698d28c7edbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:35.786811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1286/2590] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:35.790857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef4b01-4259-4a2d-8446-f322a4bacd85 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:35.800370 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef4b01-4259-4a2d-8446-f322a4bacd85 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:35.804946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcacdd3-4fe7-433e-8d64-dd372acc5a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:35.805596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcacdd3-4fe7-433e-8d64-dd372acc5a0d admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:35.805881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcacdd3-4fe7-433e-8d64-dd372acc5a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:35.806162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcacdd3-4fe7-433e-8d64-dd372acc5a0d admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:35.806922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcacdd3-4fe7-433e-8d64-dd372acc5a0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:01:35.807237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcacdd3-4fe7-433e-8d64-dd372acc5a0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:35.808841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcacdd3-4fe7-433e-8d64-dd372acc5a0d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:35.821776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1305/2591] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:01:35.825927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aa959-58c4-4ce2-b72e-698d28c7edbd None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:35.835924 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aa959-58c4-4ce2-b72e-698d28c7edbd None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:35.840607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad3467-9b5f-4f84-9edd-47810432d297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:35.841254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad3467-9b5f-4f84-9edd-47810432d297 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:35.841537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad3467-9b5f-4f84-9edd-47810432d297 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:35.841820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad3467-9b5f-4f84-9edd-47810432d297 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:35.842513 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad3467-9b5f-4f84-9edd-47810432d297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:35.844773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad3467-9b5f-4f84-9edd-47810432d297 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:35.873302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1287/2592] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:01:35 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:35.874523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcacdd3-4fe7-433e-8d64-dd372acc5a0d admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:35.885109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcacdd3-4fe7-433e-8d64-dd372acc5a0d admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:35.942454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ae626-9d30-459a-bbba-c902b672efb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:35.943113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ae626-9d30-459a-bbba-c902b672efb5 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:35.943339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ae626-9d30-459a-bbba-c902b672efb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:35.943611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ae626-9d30-459a-bbba-c902b672efb5 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:35.944785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ae626-9d30-459a-bbba-c902b672efb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:35.946686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ae626-9d30-459a-bbba-c902b672efb5 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:35.970131 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1306/2593] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:01:35 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:01:35.971282 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad3467-9b5f-4f84-9edd-47810432d297 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:35.981978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad3467-9b5f-4f84-9edd-47810432d297 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:35.987058 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09763639-c17f-4283-80ec-b6f5ecbd7b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:35.987719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763639-c17f-4283-80ec-b6f5ecbd7b67 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:35.988008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763639-c17f-4283-80ec-b6f5ecbd7b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:35.988289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763639-c17f-4283-80ec-b6f5ecbd7b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:35.989016 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09763639-c17f-4283-80ec-b6f5ecbd7b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:35.990642 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09763639-c17f-4283-80ec-b6f5ecbd7b67 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:36.001255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1288/2594] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:01:36.005421 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ae626-9d30-459a-bbba-c902b672efb5 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:36.016821 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ae626-9d30-459a-bbba-c902b672efb5 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:36.021542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d504a3-e51e-4ca7-bc4a-f9e30ca5f990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:36.022785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d504a3-e51e-4ca7-bc4a-f9e30ca5f990 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:36.023075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d504a3-e51e-4ca7-bc4a-f9e30ca5f990 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:36.023350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d504a3-e51e-4ca7-bc4a-f9e30ca5f990 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:36.024070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d504a3-e51e-4ca7-bc4a-f9e30ca5f990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:36.025658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d504a3-e51e-4ca7-bc4a-f9e30ca5f990 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:36.035458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1307/2595] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:36.040061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09763639-c17f-4283-80ec-b6f5ecbd7b67 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:36.050038 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09763639-c17f-4283-80ec-b6f5ecbd7b67 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:36.054605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c01de7-091e-4b65-9ddb-b661e002ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:36.055506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c01de7-091e-4b65-9ddb-b661e002ca06 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:36.055786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c01de7-091e-4b65-9ddb-b661e002ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:36.056060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c01de7-091e-4b65-9ddb-b661e002ca06 admin admin] PATH_INFO: `/v3/projects/4cd1cf864eca4b0eb909e812b7de3f3f/users/12a0715ba0494afd8194655dee4b80ab/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:36.071582 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c01de7-091e-4b65-9ddb-b661e002ca06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd1cf864eca4b0eb909e812b7de3f3f, user_id=12a0715ba0494afd8194655dee4b80ab, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:36.074210 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c01de7-091e-4b65-9ddb-b661e002ca06 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:36.088017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1289/2596] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:36 2022] PUT /identity/v3/projects/4cd1cf864eca4b0eb909e812b7de3f3f/users/12a0715ba0494afd8194655dee4b80ab/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:36.095282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f6027-9724-4e74-9358-1ae0cdf2f6f4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:36.095282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f6027-9724-4e74-9358-1ae0cdf2f6f4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:36.095282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f6027-9724-4e74-9358-1ae0cdf2f6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:36.112781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-e30f6027-9724-4e74-9358-1ae0cdf2f6f4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:36.114000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-e30f6027-9724-4e74-9358-1ae0cdf2f6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:36.173257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1308/2597] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:01:36.183473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c01de7-091e-4b65-9ddb-b661e002ca06 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:36.185154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-28c01de7-091e-4b65-9ddb-b661e002ca06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:36.204300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:36.244334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c01de7-091e-4b65-9ddb-b661e002ca06 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:36.247948 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94744d27-aa2e-4d8e-920f-13e8ef5818c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:36.248589 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94744d27-aa2e-4d8e-920f-13e8ef5818c4 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:36.248870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94744d27-aa2e-4d8e-920f-13e8ef5818c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:36.249145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94744d27-aa2e-4d8e-920f-13e8ef5818c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:36.258584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94744d27-aa2e-4d8e-920f-13e8ef5818c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:36.260017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94744d27-aa2e-4d8e-920f-13e8ef5818c4 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:36.269941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1290/2598] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:40.231288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37962da-b8fb-43ef-870d-00753ea1dd38 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:40.231764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37962da-b8fb-43ef-870d-00753ea1dd38 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:40.231875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37962da-b8fb-43ef-870d-00753ea1dd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:40.273895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b37962da-b8fb-43ef-870d-00753ea1dd38 None None] MFA Rules not processed for user `12a0715ba0494afd8194655dee4b80ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:40.275161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b37962da-b8fb-43ef-870d-00753ea1dd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:40.294663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:40.353086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1309/2599] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:40.354420 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94744d27-aa2e-4d8e-920f-13e8ef5818c4 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:40.365480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94744d27-aa2e-4d8e-920f-13e8ef5818c4 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:40.370407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4472c-9f97-4863-89a8-ab634e65c8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:40.371272 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4472c-9f97-4863-89a8-ab634e65c8af admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:40.371557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4472c-9f97-4863-89a8-ab634e65c8af admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:40.371836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4472c-9f97-4863-89a8-ab634e65c8af admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:40.372533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4472c-9f97-4863-89a8-ab634e65c8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:40.374432 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4472c-9f97-4863-89a8-ab634e65c8af admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:40.416532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1291/2600] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:01:40 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:01:40.420427 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37962da-b8fb-43ef-870d-00753ea1dd38 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:40.456348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37962da-b8fb-43ef-870d-00753ea1dd38 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:40.538397 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b5b65-3c46-4d64-b71c-82689dafb9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:40.538917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b5b65-3c46-4d64-b71c-82689dafb9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:40.539205 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b5b65-3c46-4d64-b71c-82689dafb9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:40.539487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b5b65-3c46-4d64-b71c-82689dafb9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:40.540194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b5b65-3c46-4d64-b71c-82689dafb9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:40.542169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b5b65-3c46-4d64-b71c-82689dafb9b3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:40.568104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1310/2601] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:01:40 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:01:40.570173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4472c-9f97-4863-89a8-ab634e65c8af admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:40.586977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4472c-9f97-4863-89a8-ab634e65c8af admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:40.593061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9c458-45f7-4748-9256-3c445361e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:40.593856 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9c458-45f7-4748-9256-3c445361e7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:40.594147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9c458-45f7-4748-9256-3c445361e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:40.594466 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9c458-45f7-4748-9256-3c445361e7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:40.595284 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9c458-45f7-4748-9256-3c445361e7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:40.597279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9c458-45f7-4748-9256-3c445361e7b0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:40.611179 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1292/2602] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:40.615867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b5b65-3c46-4d64-b71c-82689dafb9b3 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:40.628037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b5b65-3c46-4d64-b71c-82689dafb9b3 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:40.633516 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e57183-d92f-435f-b460-d3390bb883f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:40.634604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e57183-d92f-435f-b460-d3390bb883f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:40.635008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e57183-d92f-435f-b460-d3390bb883f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:40.635357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e57183-d92f-435f-b460-d3390bb883f1 admin admin] PATH_INFO: `/v3/projects/64df054a70004df3b8cc63acaedd0c2e/users/959127669a8f48198074d457f4384521/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:40.653159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e57183-d92f-435f-b460-d3390bb883f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64df054a70004df3b8cc63acaedd0c2e, user_id=959127669a8f48198074d457f4384521, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:40.655871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e57183-d92f-435f-b460-d3390bb883f1 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:40.677586 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1311/2603] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:40 2022] PUT /identity/v3/projects/64df054a70004df3b8cc63acaedd0c2e/users/959127669a8f48198074d457f4384521/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:40.679533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9c458-45f7-4748-9256-3c445361e7b0 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:40.690449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9c458-45f7-4748-9256-3c445361e7b0 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:40.752381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d2110-97f9-49b5-89be-5a269c3bad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:40.753064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d2110-97f9-49b5-89be-5a269c3bad14 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:40.753350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d2110-97f9-49b5-89be-5a269c3bad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:40.753630 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d2110-97f9-49b5-89be-5a269c3bad14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:40.754362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d2110-97f9-49b5-89be-5a269c3bad14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:40.756180 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d2110-97f9-49b5-89be-5a269c3bad14 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:40.767702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1293/2604] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:01:40.771707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e57183-d92f-435f-b460-d3390bb883f1 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:40.782864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e57183-d92f-435f-b460-d3390bb883f1 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:40.787979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b485c14-22d2-42cf-b03d-5b29e3521efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:40.788945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b485c14-22d2-42cf-b03d-5b29e3521efe admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:40.789230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b485c14-22d2-42cf-b03d-5b29e3521efe admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:40.789504 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b485c14-22d2-42cf-b03d-5b29e3521efe admin admin] PATH_INFO: `/v3/projects/64df054a70004df3b8cc63acaedd0c2e/users/959127669a8f48198074d457f4384521/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:40.792248 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b485c14-22d2-42cf-b03d-5b29e3521efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=64df054a70004df3b8cc63acaedd0c2e, user_id=959127669a8f48198074d457f4384521, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:40.795182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b485c14-22d2-42cf-b03d-5b29e3521efe admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:40.807325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1312/2605] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:01:40 2022] PUT /identity/v3/projects/64df054a70004df3b8cc63acaedd0c2e/users/959127669a8f48198074d457f4384521/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:01:45.984499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7f11d-5f80-4e64-bd02-1f50e8ac8fc5 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:45.984977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7f11d-5f80-4e64-bd02-1f50e8ac8fc5 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:45.984977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7f11d-5f80-4e64-bd02-1f50e8ac8fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:46.013752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e9b7f11d-5f80-4e64-bd02-1f50e8ac8fc5 None None] MFA Rules not processed for user `959127669a8f48198074d457f4384521`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:46.015022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e9b7f11d-5f80-4e64-bd02-1f50e8ac8fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:46.100103 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1294/2606] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:46.114590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b485c14-22d2-42cf-b03d-5b29e3521efe admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:46.116598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5b485c14-22d2-42cf-b03d-5b29e3521efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:46.189249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7f11d-5f80-4e64-bd02-1f50e8ac8fc5 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:46.190835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e9b7f11d-5f80-4e64-bd02-1f50e8ac8fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:46.205805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:46.227327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b485c14-22d2-42cf-b03d-5b29e3521efe admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:46.234382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87b0b9-ab3a-432d-a009-d20d26d3d6f7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7405c8bc8e4c84bb056cd9e1df61c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:46.235385 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87b0b9-ab3a-432d-a009-d20d26d3d6f7 service glance] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:46.235961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87b0b9-ab3a-432d-a009-d20d26d3d6f7 service glance] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:46.236462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87b0b9-ab3a-432d-a009-d20d26d3d6f7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:46.294308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:46.357625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87b0b9-ab3a-432d-a009-d20d26d3d6f7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:46.359163 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87b0b9-ab3a-432d-a009-d20d26d3d6f7 service glance] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:46.376765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7f11d-5f80-4e64-bd02-1f50e8ac8fc5 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:46.377224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1313/2607] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:46.386563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:46.387131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:46.387415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:46.387693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:46.389954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:46.408052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:46.410762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:46.432928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1295/2608] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:01:46.529984 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813a570-801d-4383-855d-6a4533daa57f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:46.530278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813a570-801d-4383-855d-6a4533daa57f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:46.530572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813a570-801d-4383-855d-6a4533daa57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:46.551491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-2813a570-801d-4383-855d-6a4533daa57f None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:46.552724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-2813a570-801d-4383-855d-6a4533daa57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:46.609082 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1314/2609] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 16:01:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:46.658143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:46.659819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:46.737379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a66073-2b68-421f-b9cf-3f95953da66e service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:46.743602 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4c79-eaf2-46b2-ab78-2a80204c8574 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:46.744257 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4c79-eaf2-46b2-ab78-2a80204c8574 service swift] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:46.744540 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4c79-eaf2-46b2-ab78-2a80204c8574 service swift] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:46.744820 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4c79-eaf2-46b2-ab78-2a80204c8574 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:46.753360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef4c79-eaf2-46b2-ab78-2a80204c8574 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:46.754976 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef4c79-eaf2-46b2-ab78-2a80204c8574 service swift] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:46.770861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1296/2610] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 16:01:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 16:01:50.489584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1f309-acb0-4224-8917-f4d6b30a9b9a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:50.490054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1f309-acb0-4224-8917-f4d6b30a9b9a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:50.490251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1f309-acb0-4224-8917-f4d6b30a9b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:50.519691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-a8e1f309-acb0-4224-8917-f4d6b30a9b9a None None] MFA Rules not processed for user `f18270589e8745a286e3c78a7f86b25d`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:50.521773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a8e1f309-acb0-4224-8917-f4d6b30a9b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:50.615967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1315/2611] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:51.157133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4c79-eaf2-46b2-ab78-2a80204c8574 service swift] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:51.223443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4c79-eaf2-46b2-ab78-2a80204c8574 service swift] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:51.229543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac393ef8-62a1-4c6f-a642-54338e8fad48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:51.230216 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac393ef8-62a1-4c6f-a642-54338e8fad48 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:51.230494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac393ef8-62a1-4c6f-a642-54338e8fad48 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:51.230805 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac393ef8-62a1-4c6f-a642-54338e8fad48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:51.232636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ac393ef8-62a1-4c6f-a642-54338e8fad48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:51.244709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac393ef8-62a1-4c6f-a642-54338e8fad48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:51.246149 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac393ef8-62a1-4c6f-a642-54338e8fad48 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:51.256383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1297/2612] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:51.900026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1f309-acb0-4224-8917-f4d6b30a9b9a None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:51.901537 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a8e1f309-acb0-4224-8917-f4d6b30a9b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:52.004647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1f309-acb0-4224-8917-f4d6b30a9b9a None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:52.012296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:52.013068 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:52.014558 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:52.015047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:52.017483 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:52.035763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:52.037809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:52.049640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1316/2613] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:52.847442 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac393ef8-62a1-4c6f-a642-54338e8fad48 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:52.857248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac393ef8-62a1-4c6f-a642-54338e8fad48 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:52.861041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab34e1-b94a-4fcc-8bd3-a324b9ea618c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:52.861946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab34e1-b94a-4fcc-8bd3-a324b9ea618c service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:52.862236 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab34e1-b94a-4fcc-8bd3-a324b9ea618c service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:52.862517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab34e1-b94a-4fcc-8bd3-a324b9ea618c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:52.864359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-82ab34e1-b94a-4fcc-8bd3-a324b9ea618c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:52.898014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:52.949421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab34e1-b94a-4fcc-8bd3-a324b9ea618c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:52.951025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab34e1-b94a-4fcc-8bd3-a324b9ea618c service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:52.961848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1298/2614] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:57.062168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:57.074564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:57.110172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=100 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:01:57.128689 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d08a45-4c5d-447e-9e6e-5ad85800b311 service placement] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:57.149145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:57.150193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:57.150896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:57.151336 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:57.153710 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:57.171799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:57.173648 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:57.216857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1317/2615] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:01:59.262315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6257dfe-7b2e-4053-9291-51a138a41ff6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:59.263523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6257dfe-7b2e-4053-9291-51a138a41ff6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:59.264012 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6257dfe-7b2e-4053-9291-51a138a41ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:59.292694 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c6257dfe-7b2e-4053-9291-51a138a41ff6 None None] MFA Rules not processed for user `12a0715ba0494afd8194655dee4b80ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:01:59.294523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c6257dfe-7b2e-4053-9291-51a138a41ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:59.310772 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1299/2616] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:01:59.888918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:59.890957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:59.904279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ef9d6-3beb-4794-8809-61269a6c6bbf service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:59.904723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6257dfe-7b2e-4053-9291-51a138a41ff6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:01:59.908809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:59.909876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:59.910155 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:59.910430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] PATH_INFO: `/v3/users/fb838f931b7748e2baf6c2315d3cbd75` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:59.914136 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb838f931b7748e2baf6c2315d3cbd75)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:59.916144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:59.916333 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6257dfe-7b2e-4053-9291-51a138a41ff6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:01:59.920596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee449d-66c5-43de-afd0-bcae8f22cb70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:01:59.921894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee449d-66c5-43de-afd0-bcae8f22cb70 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:01:59.921894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee449d-66c5-43de-afd0-bcae8f22cb70 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:01:59.922085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee449d-66c5-43de-afd0-bcae8f22cb70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:01:59.924040 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0fee449d-66c5-43de-afd0-bcae8f22cb70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:01:59.937484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee449d-66c5-43de-afd0-bcae8f22cb70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:01:59.939852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee449d-66c5-43de-afd0-bcae8f22cb70 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:01:59.949619 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1300/2617] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:01:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:01:59.973472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb838f931b7748e2baf6c2315d3cbd75'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:59.981806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb838f931b7748e2baf6c2315d3cbd75'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:59.989088 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb838f931b7748e2baf6c2315d3cbd75'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:01:59.990244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb838f931b7748e2baf6c2315d3cbd75'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:00.003318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-019ca4ed-fa29-4054-882a-b2882aa697a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb838f931b7748e2baf6c2315d3cbd75'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:00.009311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1318/2618] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:01:59 2022] DELETE /identity/v3/users/fb838f931b7748e2baf6c2315d3cbd75 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:00.192538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee449d-66c5-43de-afd0-bcae8f22cb70 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:00.208167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee449d-66c5-43de-afd0-bcae8f22cb70 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:00.290073 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:00.291081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.291478 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.291830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] PATH_INFO: `/v3/projects/0496c4ee9cdf486684000904876d0b6b` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.293189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0496c4ee9cdf486684000904876d0b6b)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:00.306621 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:00.385474 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0496c4ee9cdf486684000904876d0b6b', 'request_id': 'req-e5482493-9592-4df2-acd0-50bcfbeb771f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:00.406354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0496c4ee9cdf486684000904876d0b6b', 'request_id': 'req-e5482493-9592-4df2-acd0-50bcfbeb771f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:00.423875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0496c4ee9cdf486684000904876d0b6b', 'request_id': 'req-e5482493-9592-4df2-acd0-50bcfbeb771f'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:00.456726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] The token cache is being invalidate because project 0496c4ee9cdf486684000904876d0b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:00.457022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:00.458066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:00.460839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1301/2619] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:00 2022] DELETE /identity/v3/projects/0496c4ee9cdf486684000904876d0b6b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:00.466466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7454de-819c-4204-b701-db6a4633412a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.466743 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7454de-819c-4204-b701-db6a4633412a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.467199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7454de-819c-4204-b701-db6a4633412a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.498723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:00.500256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:00.502096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-1c7454de-819c-4204-b701-db6a4633412a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:00.503293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1c7454de-819c-4204-b701-db6a4633412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:00.542108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:00.574798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5482493-9592-4df2-acd0-50bcfbeb771f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:00.578563 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1319/2620] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:00.578930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3877364-33ad-4116-a981-b454752eb724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:00.579657 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3877364-33ad-4116-a981-b454752eb724 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.580004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3877364-33ad-4116-a981-b454752eb724 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.580293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3877364-33ad-4116-a981-b454752eb724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.582178 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c3877364-33ad-4116-a981-b454752eb724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:00.587067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b83569-6272-42c4-bc3b-fc61bc8806b9 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.587597 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b83569-6272-42c4-bc3b-fc61bc8806b9 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.588099 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b83569-6272-42c4-bc3b-fc61bc8806b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.596008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3877364-33ad-4116-a981-b454752eb724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:00.597448 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3877364-33ad-4116-a981-b454752eb724 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:00.608342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1302/2621] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:02:00.614795 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-68b83569-6272-42c4-bc3b-fc61bc8806b9 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:00.616321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-68b83569-6272-42c4-bc3b-fc61bc8806b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:00.625475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1320/2622] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:00.629828 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3877364-33ad-4116-a981-b454752eb724 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:00.642193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3877364-33ad-4116-a981-b454752eb724 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:00.646934 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed7693-127f-4edc-b416-b713e2fd46f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:00.648161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed7693-127f-4edc-b416-b713e2fd46f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.648447 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed7693-127f-4edc-b416-b713e2fd46f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.648724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed7693-127f-4edc-b416-b713e2fd46f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.649443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed7693-127f-4edc-b416-b713e2fd46f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:02:00.649752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed7693-127f-4edc-b416-b713e2fd46f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:00.651377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed7693-127f-4edc-b416-b713e2fd46f1 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:00.664969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1303/2623] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:02:00.669665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b83569-6272-42c4-bc3b-fc61bc8806b9 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:00.679138 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b83569-6272-42c4-bc3b-fc61bc8806b9 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:00.686447 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c8afb-662c-421a-a007-d26ea6b29820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:00.687223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c8afb-662c-421a-a007-d26ea6b29820 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.687582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c8afb-662c-421a-a007-d26ea6b29820 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.687949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c8afb-662c-421a-a007-d26ea6b29820 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.688760 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c8afb-662c-421a-a007-d26ea6b29820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:00.690670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c8afb-662c-421a-a007-d26ea6b29820 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:00.722115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1321/2624] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:00 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:00.726233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed7693-127f-4edc-b416-b713e2fd46f1 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:00.736681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed7693-127f-4edc-b416-b713e2fd46f1 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:00.802110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2908a-5568-4c17-b355-291fadc1c128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:00.802837 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2908a-5568-4c17-b355-291fadc1c128 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.803124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2908a-5568-4c17-b355-291fadc1c128 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.803405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2908a-5568-4c17-b355-291fadc1c128 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.804140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2908a-5568-4c17-b355-291fadc1c128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:00.806056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2908a-5568-4c17-b355-291fadc1c128 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:00.830618 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1304/2625] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:00 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:00.835791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c8afb-662c-421a-a007-d26ea6b29820 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:00.848090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c8afb-662c-421a-a007-d26ea6b29820 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:00.853476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbaec4-1856-4de8-bcf5-93839e685b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:00.854228 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbaec4-1856-4de8-bcf5-93839e685b36 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.854582 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbaec4-1856-4de8-bcf5-93839e685b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.855091 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbaec4-1856-4de8-bcf5-93839e685b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.855897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbaec4-1856-4de8-bcf5-93839e685b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:00.857591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbaec4-1856-4de8-bcf5-93839e685b36 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:00.870842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1322/2626] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:00.874909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2908a-5568-4c17-b355-291fadc1c128 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:00.885891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2908a-5568-4c17-b355-291fadc1c128 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:00.892083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b9215-73b3-410d-9a2a-da1465b83fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:00.892926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b9215-73b3-410d-9a2a-da1465b83fda admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.893218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b9215-73b3-410d-9a2a-da1465b83fda admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.893569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b9215-73b3-410d-9a2a-da1465b83fda admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.894409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b9215-73b3-410d-9a2a-da1465b83fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:00.896098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b9215-73b3-410d-9a2a-da1465b83fda admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:00.909371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1305/2627] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:00.913620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbaec4-1856-4de8-bcf5-93839e685b36 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:00.926240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbaec4-1856-4de8-bcf5-93839e685b36 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:00.935732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818148e-c127-410e-a18e-b36b306213d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:00.936952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818148e-c127-410e-a18e-b36b306213d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.937320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818148e-c127-410e-a18e-b36b306213d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.937687 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818148e-c127-410e-a18e-b36b306213d7 admin admin] PATH_INFO: `/v3/projects/8c80f76984fa4509a5629943bfd5265c/users/4465706fd66348f882712834cbf2921e/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:00.958496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818148e-c127-410e-a18e-b36b306213d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c80f76984fa4509a5629943bfd5265c, user_id=4465706fd66348f882712834cbf2921e, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:00.961348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818148e-c127-410e-a18e-b36b306213d7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:00.973317 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:00.984067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1323/2628] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:00 2022] PUT /identity/v3/projects/8c80f76984fa4509a5629943bfd5265c/users/4465706fd66348f882712834cbf2921e/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:00.985784 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3fbb2-48ec-40e9-b4ea-d038bb687019 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:00.986059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3fbb2-48ec-40e9-b4ea-d038bb687019 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:00.986338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3fbb2-48ec-40e9-b4ea-d038bb687019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:01.006048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-a1f3fbb2-48ec-40e9-b4ea-d038bb687019 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:01.007860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a1f3fbb2-48ec-40e9-b4ea-d038bb687019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:01.076933 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1306/2629] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:01.086414 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818148e-c127-410e-a18e-b36b306213d7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:01.088170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8818148e-c127-410e-a18e-b36b306213d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:01.147543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818148e-c127-410e-a18e-b36b306213d7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:01.151134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac5149-a0fc-4d80-82ec-b0214ff2cd8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:01.151776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac5149-a0fc-4d80-82ec-b0214ff2cd8a service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:01.152051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac5149-a0fc-4d80-82ec-b0214ff2cd8a service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:01.152324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac5149-a0fc-4d80-82ec-b0214ff2cd8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:01.161544 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac5149-a0fc-4d80-82ec-b0214ff2cd8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:01.163408 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac5149-a0fc-4d80-82ec-b0214ff2cd8a service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:01.172831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1324/2630] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:01.818955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0559e8-64b6-4f96-b6cd-c1a615854e21 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:01.819488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0559e8-64b6-4f96-b6cd-c1a615854e21 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:01.819601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0559e8-64b6-4f96-b6cd-c1a615854e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:01.845375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0d0559e8-64b6-4f96-b6cd-c1a615854e21 None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:01.846836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0d0559e8-64b6-4f96-b6cd-c1a615854e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:01.898634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:01.934895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1307/2631] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Jun 15 16:02:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:01.948000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac5149-a0fc-4d80-82ec-b0214ff2cd8a service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:01.951254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a7ac5149-a0fc-4d80-82ec-b0214ff2cd8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:02.041194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac5149-a0fc-4d80-82ec-b0214ff2cd8a service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:02.049489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ddd5f-025d-4c31-8286-34c1b832517c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:02.051073 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ddd5f-025d-4c31-8286-34c1b832517c service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:02.051524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ddd5f-025d-4c31-8286-34c1b832517c service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:02.051978 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ddd5f-025d-4c31-8286-34c1b832517c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:02.062630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ddd5f-025d-4c31-8286-34c1b832517c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:02.064498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ddd5f-025d-4c31-8286-34c1b832517c service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:02.074770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1325/2632] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 16:02:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 16:02:02.517774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07ed08-cb26-44f7-a1ce-e149c3c98dd3 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:02.518077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07ed08-cb26-44f7-a1ce-e149c3c98dd3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:02.518397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07ed08-cb26-44f7-a1ce-e149c3c98dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:02.598680 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8a07ed08-cb26-44f7-a1ce-e149c3c98dd3 None None] MFA Rules not processed for user `4465706fd66348f882712834cbf2921e`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:02.600806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8a07ed08-cb26-44f7-a1ce-e149c3c98dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:02.811557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1308/2633] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:02.812780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ddd5f-025d-4c31-8286-34c1b832517c service swift] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:02.841275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ddd5f-025d-4c31-8286-34c1b832517c service swift] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:02.864451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e1a79-17dc-4e43-a5c6-6872420b7599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:02.865269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e1a79-17dc-4e43-a5c6-6872420b7599 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:02.865584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e1a79-17dc-4e43-a5c6-6872420b7599 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:02.865753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e1a79-17dc-4e43-a5c6-6872420b7599 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:02.867097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e1a79-17dc-4e43-a5c6-6872420b7599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:02.874701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e1a79-17dc-4e43-a5c6-6872420b7599 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:02.975322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1326/2634] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:02 2022] POST /identity/v3/projects => generated 405 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:02.980291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07ed08-cb26-44f7-a1ce-e149c3c98dd3 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:03.003875 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07ed08-cb26-44f7-a1ce-e149c3c98dd3 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:03.173745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4d5c7-b408-4df8-a5b8-e169f1e3d9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:03.174548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4d5c7-b408-4df8-a5b8-e169f1e3d9cd admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:03.176251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4d5c7-b408-4df8-a5b8-e169f1e3d9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:03.176634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4d5c7-b408-4df8-a5b8-e169f1e3d9cd admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:03.189218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4d5c7-b408-4df8-a5b8-e169f1e3d9cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:03.196607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4d5c7-b408-4df8-a5b8-e169f1e3d9cd admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:03.258130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1309/2635] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:02 2022] POST /identity/v3/users => generated 309 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:03.258614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e1a79-17dc-4e43-a5c6-6872420b7599 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:03.277508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e1a79-17dc-4e43-a5c6-6872420b7599 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:03.300522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee043f-19f9-4b5e-a65a-d63219accb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:03.301465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee043f-19f9-4b5e-a65a-d63219accb30 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:03.302037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee043f-19f9-4b5e-a65a-d63219accb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:03.302489 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee043f-19f9-4b5e-a65a-d63219accb30 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:03.311434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee043f-19f9-4b5e-a65a-d63219accb30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:03.313574 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee043f-19f9-4b5e-a65a-d63219accb30 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:03.343671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1327/2636] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:03.346654 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4d5c7-b408-4df8-a5b8-e169f1e3d9cd admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:03.357983 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4d5c7-b408-4df8-a5b8-e169f1e3d9cd admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:03.363972 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393f38d-b905-4649-add7-a3b384331f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:03.365017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393f38d-b905-4649-add7-a3b384331f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:03.365364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393f38d-b905-4649-add7-a3b384331f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:03.365752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393f38d-b905-4649-add7-a3b384331f30 admin admin] PATH_INFO: `/v3/projects/87ccf25e22b34f47ad470fee56134bc7/users/5cda0554e8bf45b08d1d680cef183437/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:03.418081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393f38d-b905-4649-add7-a3b384331f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ccf25e22b34f47ad470fee56134bc7, user_id=5cda0554e8bf45b08d1d680cef183437, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:03.420911 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393f38d-b905-4649-add7-a3b384331f30 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:03.444962 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1310/2637] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:03 2022] PUT /identity/v3/projects/87ccf25e22b34f47ad470fee56134bc7/users/5cda0554e8bf45b08d1d680cef183437/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:03.449386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee043f-19f9-4b5e-a65a-d63219accb30 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:03.470054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee043f-19f9-4b5e-a65a-d63219accb30 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:03.614005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38385e7f-a30a-47c3-99fb-58756ec46023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:03.614947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385e7f-a30a-47c3-99fb-58756ec46023 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:03.615331 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385e7f-a30a-47c3-99fb-58756ec46023 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:03.615699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385e7f-a30a-47c3-99fb-58756ec46023 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:03.616901 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38385e7f-a30a-47c3-99fb-58756ec46023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:03.618772 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38385e7f-a30a-47c3-99fb-58756ec46023 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:03.624862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:03.649898 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393f38d-b905-4649-add7-a3b384331f30 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:03.655981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1328/2638] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:03.661700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393f38d-b905-4649-add7-a3b384331f30 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:03.678374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d68a8-3120-43dd-b119-ed478b27a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:03.679423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d68a8-3120-43dd-b119-ed478b27a571 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:03.679766 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d68a8-3120-43dd-b119-ed478b27a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:03.680131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d68a8-3120-43dd-b119-ed478b27a571 admin admin] PATH_INFO: `/v3/projects/87ccf25e22b34f47ad470fee56134bc7/users/5cda0554e8bf45b08d1d680cef183437/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:03.687651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d68a8-3120-43dd-b119-ed478b27a571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ccf25e22b34f47ad470fee56134bc7, user_id=5cda0554e8bf45b08d1d680cef183437, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:03.690586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d68a8-3120-43dd-b119-ed478b27a571 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:03.719022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1311/2639] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:03 2022] PUT /identity/v3/projects/87ccf25e22b34f47ad470fee56134bc7/users/5cda0554e8bf45b08d1d680cef183437/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:05.355778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9a453-4e7e-4d6a-909b-49e4631f60ac None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:05.356852 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9a453-4e7e-4d6a-909b-49e4631f60ac None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:05.357359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9a453-4e7e-4d6a-909b-49e4631f60ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:05.449329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-f2b9a453-4e7e-4d6a-909b-49e4631f60ac None None] MFA Rules not processed for user `5cda0554e8bf45b08d1d680cef183437`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:05.452093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f2b9a453-4e7e-4d6a-909b-49e4631f60ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:05.638142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1329/2640] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:05 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:05.864407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d68a8-3120-43dd-b119-ed478b27a571 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:05.867661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-606d68a8-3120-43dd-b119-ed478b27a571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:05.898635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:05.939804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d68a8-3120-43dd-b119-ed478b27a571 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:05.944315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:05.945189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:05.945476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:05.945755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] PATH_INFO: `/v3/users/3af6df3efbb74db4958654503e17d82e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:05.947166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3af6df3efbb74db4958654503e17d82e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:05.949226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:05.998926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3af6df3efbb74db4958654503e17d82e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.006526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3af6df3efbb74db4958654503e17d82e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.013853 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3af6df3efbb74db4958654503e17d82e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.014508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3af6df3efbb74db4958654503e17d82e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.015134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-7b7b0f45-ca56-475f-b556-c1f17c6fb9b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3af6df3efbb74db4958654503e17d82e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.032824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1312/2641] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:05 2022] DELETE /identity/v3/users/3af6df3efbb74db4958654503e17d82e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:06.233160 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9a453-4e7e-4d6a-909b-49e4631f60ac None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:06.251366 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9a453-4e7e-4d6a-909b-49e4631f60ac None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:06.336630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:06.338112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:06.338426 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:06.339613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] PATH_INFO: `/v3/projects/9e7b007aaf80436f8fa691a12a95f21e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:06.341035 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7b007aaf80436f8fa691a12a95f21e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:06.343729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:06.414528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7b007aaf80436f8fa691a12a95f21e', 'request_id': 'req-91e13e74-843e-4c29-9485-e998389f87ed'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.421321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7b007aaf80436f8fa691a12a95f21e', 'request_id': 'req-91e13e74-843e-4c29-9485-e998389f87ed'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.429692 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7b007aaf80436f8fa691a12a95f21e', 'request_id': 'req-91e13e74-843e-4c29-9485-e998389f87ed'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.456574 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] The token cache is being invalidate because project 9e7b007aaf80436f8fa691a12a95f21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:06.456876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.457532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-91e13e74-843e-4c29-9485-e998389f87ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:06.462850 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1330/2642] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:06 2022] DELETE /identity/v3/projects/9e7b007aaf80436f8fa691a12a95f21e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:06.465473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d0ea2-5c94-4d35-a704-95239e6669e3 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:06.465834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d0ea2-5c94-4d35-a704-95239e6669e3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:06.466179 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d0ea2-5c94-4d35-a704-95239e6669e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:06.489297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-609d0ea2-5c94-4d35-a704-95239e6669e3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:06.490585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-609d0ea2-5c94-4d35-a704-95239e6669e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:06.599859 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1313/2643] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:06.609039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f2dd2-3d87-4da6-9912-b8d0b7f35e26 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:06.609350 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f2dd2-3d87-4da6-9912-b8d0b7f35e26 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:06.609655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f2dd2-3d87-4da6-9912-b8d0b7f35e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:06.638927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-0a3f2dd2-3d87-4da6-9912-b8d0b7f35e26 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:06.639858 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0a3f2dd2-3d87-4da6-9912-b8d0b7f35e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:06.651455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1331/2644] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:06.655845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d0ea2-5c94-4d35-a704-95239e6669e3 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:06.670400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d0ea2-5c94-4d35-a704-95239e6669e3 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:06.684898 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a0f1-c994-465d-940d-eed65cd79be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:06.685926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a0f1-c994-465d-940d-eed65cd79be5 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:06.686395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a0f1-c994-465d-940d-eed65cd79be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:06.686908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a0f1-c994-465d-940d-eed65cd79be5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:06.687865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34a0f1-c994-465d-940d-eed65cd79be5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:02:06.688398 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34a0f1-c994-465d-940d-eed65cd79be5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:06.690683 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34a0f1-c994-465d-940d-eed65cd79be5 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:06.723287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1314/2645] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:02:06.727465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f2dd2-3d87-4da6-9912-b8d0b7f35e26 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:06.737237 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:06.743333 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f2dd2-3d87-4da6-9912-b8d0b7f35e26 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:06.763067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c44c49-55b5-4a25-8f97-c297bc28a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:06.763855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c44c49-55b5-4a25-8f97-c297bc28a13d admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:06.764309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c44c49-55b5-4a25-8f97-c297bc28a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:06.764630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c44c49-55b5-4a25-8f97-c297bc28a13d admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:06.765778 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c44c49-55b5-4a25-8f97-c297bc28a13d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:06.768048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c44c49-55b5-4a25-8f97-c297bc28a13d admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:06.825552 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a0f1-c994-465d-940d-eed65cd79be5 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:06.825880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1332/2646] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:06 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:02:06.844000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a0f1-c994-465d-940d-eed65cd79be5 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:06.929424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:06.970483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e63ac-1f6d-4c22-b9a2-0eff2eee4aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:06.971544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e63ac-1f6d-4c22-b9a2-0eff2eee4aaa admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:06.971847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e63ac-1f6d-4c22-b9a2-0eff2eee4aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:06.972213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e63ac-1f6d-4c22-b9a2-0eff2eee4aaa admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:06.973059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e63ac-1f6d-4c22-b9a2-0eff2eee4aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:06.976810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e63ac-1f6d-4c22-b9a2-0eff2eee4aaa admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:07.004538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1315/2647] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:06 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:02:07.005958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c44c49-55b5-4a25-8f97-c297bc28a13d admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:07.023017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c44c49-55b5-4a25-8f97-c297bc28a13d admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:07.029207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f8068-bb13-4b6b-b404-d0a84c6d1281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:07.030195 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f8068-bb13-4b6b-b404-d0a84c6d1281 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:07.030629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f8068-bb13-4b6b-b404-d0a84c6d1281 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:07.030866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f8068-bb13-4b6b-b404-d0a84c6d1281 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:07.031721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f8068-bb13-4b6b-b404-d0a84c6d1281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:07.035971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f8068-bb13-4b6b-b404-d0a84c6d1281 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:07.050799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1333/2648] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:07.052263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e63ac-1f6d-4c22-b9a2-0eff2eee4aaa admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:07.065719 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e63ac-1f6d-4c22-b9a2-0eff2eee4aaa admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:07.071369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03503e-c65f-4def-98b2-d2f50360e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:07.072206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03503e-c65f-4def-98b2-d2f50360e2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:07.072490 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03503e-c65f-4def-98b2-d2f50360e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:07.072770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03503e-c65f-4def-98b2-d2f50360e2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:07.073867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03503e-c65f-4def-98b2-d2f50360e2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:07.075914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03503e-c65f-4def-98b2-d2f50360e2b2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:07.090155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1316/2649] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:07.091620 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f8068-bb13-4b6b-b404-d0a84c6d1281 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:07.111779 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f8068-bb13-4b6b-b404-d0a84c6d1281 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:07.123165 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba968e-15a2-4bc7-939d-02145df05fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:07.124353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba968e-15a2-4bc7-939d-02145df05fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:07.124719 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba968e-15a2-4bc7-939d-02145df05fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:07.124998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba968e-15a2-4bc7-939d-02145df05fd2 admin admin] PATH_INFO: `/v3/projects/ee07a0779fa943ba93c48c46ede1ca25/users/9af273a454674517bcb4d8c3e043c094/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:07.165536 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba968e-15a2-4bc7-939d-02145df05fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee07a0779fa943ba93c48c46ede1ca25, user_id=9af273a454674517bcb4d8c3e043c094, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:07.168900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba968e-15a2-4bc7-939d-02145df05fd2 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:07.192532 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1334/2650] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:07 2022] PUT /identity/v3/projects/ee07a0779fa943ba93c48c46ede1ca25/users/9af273a454674517bcb4d8c3e043c094/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:07.207562 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c7a6-0181-45d1-9118-a99876fd63a4 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:07.208033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c7a6-0181-45d1-9118-a99876fd63a4 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:07.208377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c7a6-0181-45d1-9118-a99876fd63a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:07.253274 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-83a0c7a6-0181-45d1-9118-a99876fd63a4 None None] MFA Rules not processed for user `9af273a454674517bcb4d8c3e043c094`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:07.272866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-83a0c7a6-0181-45d1-9118-a99876fd63a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:07.402925 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1317/2651] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:07.407075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba968e-15a2-4bc7-939d-02145df05fd2 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:07.495558 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba968e-15a2-4bc7-939d-02145df05fd2 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:07.504096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e6c2c-c91a-4811-8db9-53be8b5b7908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:07.505027 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e6c2c-c91a-4811-8db9-53be8b5b7908 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:07.505459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e6c2c-c91a-4811-8db9-53be8b5b7908 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:07.505893 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e6c2c-c91a-4811-8db9-53be8b5b7908 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:07.506999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e6c2c-c91a-4811-8db9-53be8b5b7908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:07.509227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e6c2c-c91a-4811-8db9-53be8b5b7908 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:07.553574 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1335/2652] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:07 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:07.555324 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0c7a6-0181-45d1-9118-a99876fd63a4 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:07.568795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0c7a6-0181-45d1-9118-a99876fd63a4 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:07.714762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b408e4-f161-4972-96cc-d5f43bf14aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:07.716027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b408e4-f161-4972-96cc-d5f43bf14aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:07.719693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b408e4-f161-4972-96cc-d5f43bf14aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:07.719693 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b408e4-f161-4972-96cc-d5f43bf14aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:07.721455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b408e4-f161-4972-96cc-d5f43bf14aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:07.723990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b408e4-f161-4972-96cc-d5f43bf14aa0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:07.771609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1318/2653] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:07 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:07.775734 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e6c2c-c91a-4811-8db9-53be8b5b7908 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:07.787155 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b408e4-f161-4972-96cc-d5f43bf14aa0 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:07.790887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-46b408e4-f161-4972-96cc-d5f43bf14aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:07.800534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e6c2c-c91a-4811-8db9-53be8b5b7908 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:07.810991 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799a0d9-bec6-46cf-afec-4163eba22093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:07.811598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799a0d9-bec6-46cf-afec-4163eba22093 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:07.812006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799a0d9-bec6-46cf-afec-4163eba22093 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:07.812221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799a0d9-bec6-46cf-afec-4163eba22093 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:07.815705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799a0d9-bec6-46cf-afec-4163eba22093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:07.815705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799a0d9-bec6-46cf-afec-4163eba22093 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:07.833517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b408e4-f161-4972-96cc-d5f43bf14aa0 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:07.842791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1336/2654] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:07.847017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799a0d9-bec6-46cf-afec-4163eba22093 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:07.849863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:07.859786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:07.860895 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:07.861197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] PATH_INFO: `/v3/users/4465706fd66348f882712834cbf2921e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:07.862931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4465706fd66348f882712834cbf2921e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:07.864967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799a0d9-bec6-46cf-afec-4163eba22093 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:07.870189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:07.871655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4351c-9c41-4204-8964-f4cf9884d58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:07.872811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4351c-9c41-4204-8964-f4cf9884d58a admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:07.873203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4351c-9c41-4204-8964-f4cf9884d58a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:07.873638 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4351c-9c41-4204-8964-f4cf9884d58a admin admin] PATH_INFO: `/v3/projects/d1461bf3e65b440492e4bfb08a781c01/users/7a5f6d1f04b0408090acc5e3ca72bc78/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:07.895251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4351c-9c41-4204-8964-f4cf9884d58a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1461bf3e65b440492e4bfb08a781c01, user_id=7a5f6d1f04b0408090acc5e3ca72bc78, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:07.898764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4351c-9c41-4204-8964-f4cf9884d58a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:07.920334 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1337/2655] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:07 2022] PUT /identity/v3/projects/d1461bf3e65b440492e4bfb08a781c01/users/7a5f6d1f04b0408090acc5e3ca72bc78/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:07.925892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4351c-9c41-4204-8964-f4cf9884d58a admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:07.946867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:08.013597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4465706fd66348f882712834cbf2921e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.044374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4465706fd66348f882712834cbf2921e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.055239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:08.058677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4465706fd66348f882712834cbf2921e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.059786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4465706fd66348f882712834cbf2921e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.060347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4465706fd66348f882712834cbf2921e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.071957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4351c-9c41-4204-8964-f4cf9884d58a admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:08.086215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b56d5-39be-41cc-b060-c967e1e74d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:08.087606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b56d5-39be-41cc-b060-c967e1e74d79 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:08.088209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b56d5-39be-41cc-b060-c967e1e74d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:08.089008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b56d5-39be-41cc-b060-c967e1e74d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:08.090611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b56d5-39be-41cc-b060-c967e1e74d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:08.092371 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b56d5-39be-41cc-b060-c967e1e74d79 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:08.097327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1319/2656] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:07 2022] DELETE /identity/v3/users/4465706fd66348f882712834cbf2921e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:08.133284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1338/2657] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:08.137083 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:08.138565 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:08.231506 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd180da-bd45-404c-bc55-f4e9c7a71ac0 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:08.244169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82678515-3311-4a92-8840-e3f3096dcf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:08.246184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82678515-3311-4a92-8840-e3f3096dcf0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:08.246484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82678515-3311-4a92-8840-e3f3096dcf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:08.249279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82678515-3311-4a92-8840-e3f3096dcf0f admin admin] PATH_INFO: `/v3/projects/d1461bf3e65b440492e4bfb08a781c01/users/7a5f6d1f04b0408090acc5e3ca72bc78/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:08.250646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82678515-3311-4a92-8840-e3f3096dcf0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1461bf3e65b440492e4bfb08a781c01, user_id=7a5f6d1f04b0408090acc5e3ca72bc78, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:08.253423 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82678515-3311-4a92-8840-e3f3096dcf0f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:08.273064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1320/2658] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:08 2022] PUT /identity/v3/projects/d1461bf3e65b440492e4bfb08a781c01/users/7a5f6d1f04b0408090acc5e3ca72bc78/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:08.286260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96a8c-02ed-439e-84f8-2afd1ac3f1f4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:08.286549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96a8c-02ed-439e-84f8-2afd1ac3f1f4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:08.286854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96a8c-02ed-439e-84f8-2afd1ac3f1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:08.318266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5df96a8c-02ed-439e-84f8-2afd1ac3f1f4 None None] MFA Rules not processed for user `7a5f6d1f04b0408090acc5e3ca72bc78`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:08.320070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5df96a8c-02ed-439e-84f8-2afd1ac3f1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:08.404743 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82678515-3311-4a92-8840-e3f3096dcf0f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:08.407300 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-82678515-3311-4a92-8840-e3f3096dcf0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:08.428283 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1339/2659] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:08.453094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96a8c-02ed-439e-84f8-2afd1ac3f1f4 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:08.455190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5df96a8c-02ed-439e-84f8-2afd1ac3f1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:08.520308 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96a8c-02ed-439e-84f8-2afd1ac3f1f4 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:08.521003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82678515-3311-4a92-8840-e3f3096dcf0f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:08.527033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ba47c-e746-4ab3-b1bd-bded58ac5bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:08.527033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ba47c-e746-4ab3-b1bd-bded58ac5bdc service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:08.527033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ba47c-e746-4ab3-b1bd-bded58ac5bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:08.527033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ba47c-e746-4ab3-b1bd-bded58ac5bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:08.530337 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:08.531339 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:08.531844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:08.532196 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] PATH_INFO: `/v3/projects/8c80f76984fa4509a5629943bfd5265c` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:08.534653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c80f76984fa4509a5629943bfd5265c)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:08.536239 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ba47c-e746-4ab3-b1bd-bded58ac5bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:08.537174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:08.537957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ba47c-e746-4ab3-b1bd-bded58ac5bdc service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:08.548881 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1340/2660] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:08.593409 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:08.617277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c80f76984fa4509a5629943bfd5265c', 'request_id': 'req-05284086-a93d-40a4-ab4b-e2c135b6aa06'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.627454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c80f76984fa4509a5629943bfd5265c', 'request_id': 'req-05284086-a93d-40a4-ab4b-e2c135b6aa06'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.640027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c80f76984fa4509a5629943bfd5265c', 'request_id': 'req-05284086-a93d-40a4-ab4b-e2c135b6aa06'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.661243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] The token cache is being invalidate because project 8c80f76984fa4509a5629943bfd5265c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:08.661882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.664306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.667792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1321/2661] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:08 2022] DELETE /identity/v3/projects/8c80f76984fa4509a5629943bfd5265c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:08.671950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ba47c-e746-4ab3-b1bd-bded58ac5bdc service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:08.690298 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ba47c-e746-4ab3-b1bd-bded58ac5bdc service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:08.844072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:08.845269 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:08.845480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:08.845875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] PATH_INFO: `/v3/users/5cda0554e8bf45b08d1d680cef183437` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:08.847394 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cda0554e8bf45b08d1d680cef183437)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:08.849584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:08.877731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:08.945252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cda0554e8bf45b08d1d680cef183437'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.964912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cda0554e8bf45b08d1d680cef183437'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.976273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cda0554e8bf45b08d1d680cef183437'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:08.977410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cda0554e8bf45b08d1d680cef183437'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.017251 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cda0554e8bf45b08d1d680cef183437'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.028529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1341/2662] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:08 2022] DELETE /identity/v3/users/5cda0554e8bf45b08d1d680cef183437 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:09.132572 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:09.134539 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:09.211777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05284086-a93d-40a4-ab4b-e2c135b6aa06 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:09.304005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a5f6d1f04b0408090acc5e3ca72bc78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1461bf3e65b440492e4bfb08a781c01', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:09.305317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:09.305855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:09.306369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:09.307421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:02:09.307979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:09.309836 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:09.324394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1322/2663] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:02:09.328602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:09.330064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:09.381204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:09.381204 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:09.391610 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9b08d-7b9c-43ad-89c3-58e71e603c72 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:09.411046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0b9a-501c-4f89-9bf4-5dda4b3730cf tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a5f6d1f04b0408090acc5e3ca72bc78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1461bf3e65b440492e4bfb08a781c01', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:09.411046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0b9a-501c-4f89-9bf4-5dda4b3730cf tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:09.411046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0b9a-501c-4f89-9bf4-5dda4b3730cf tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:09.413139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0b9a-501c-4f89-9bf4-5dda4b3730cf tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:09.413139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc0b9a-501c-4f89-9bf4-5dda4b3730cf tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:09.413139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc0b9a-501c-4f89-9bf4-5dda4b3730cf tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:09.519962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1342/2664] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:09 2022] POST /identity/v3/projects => generated 354 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:09.521043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0b9a-501c-4f89-9bf4-5dda4b3730cf tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:09.585834 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213e7a8-3f43-4841-abd5-4bf21f7b489e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:09.597191 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:09.642310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0b9a-501c-4f89-9bf4-5dda4b3730cf tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:09.649589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b468de9-d5d5-4e4c-8358-e1ace5a33c7e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a5f6d1f04b0408090acc5e3ca72bc78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1461bf3e65b440492e4bfb08a781c01', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:09.650446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b468de9-d5d5-4e4c-8358-e1ace5a33c7e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:09.650905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b468de9-d5d5-4e4c-8358-e1ace5a33c7e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:09.651196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b468de9-d5d5-4e4c-8358-e1ace5a33c7e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:09.652212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b468de9-d5d5-4e4c-8358-e1ace5a33c7e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:02:09.652665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b468de9-d5d5-4e4c-8358-e1ace5a33c7e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:09.655186 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b468de9-d5d5-4e4c-8358-e1ace5a33c7e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:09.677292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1343/2665] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:02:09.699945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:09.700941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:09.701329 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:09.701709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] PATH_INFO: `/v3/projects/87ccf25e22b34f47ad470fee56134bc7` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:09.703205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ccf25e22b34f47ad470fee56134bc7)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:09.705590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:09.722130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b468de9-d5d5-4e4c-8358-e1ace5a33c7e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:09.734431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b468de9-d5d5-4e4c-8358-e1ace5a33c7e tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:09.750757 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a5f6d1f04b0408090acc5e3ca72bc78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1461bf3e65b440492e4bfb08a781c01', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:09.751885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:09.752252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:09.752615 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] PATH_INFO: `/v3/projects/dc4f86f69ce745d79bd3d7ea5e9d061c` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:09.754818 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dc4f86f69ce745d79bd3d7ea5e9d061c)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:09.759756 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:09.800003 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ccf25e22b34f47ad470fee56134bc7', 'request_id': 'req-56a67c54-fb9d-45be-a788-da94ca7cb2ed'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.814247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ccf25e22b34f47ad470fee56134bc7', 'request_id': 'req-56a67c54-fb9d-45be-a788-da94ca7cb2ed'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.824028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ccf25e22b34f47ad470fee56134bc7', 'request_id': 'req-56a67c54-fb9d-45be-a788-da94ca7cb2ed'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.861970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] The token cache is being invalidate because project 87ccf25e22b34f47ad470fee56134bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:09.862455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.875248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-56a67c54-fb9d-45be-a788-da94ca7cb2ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.881597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1323/2666] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:09 2022] DELETE /identity/v3/projects/87ccf25e22b34f47ad470fee56134bc7 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:09.883410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ebd1a-6b13-469c-b64f-54b58bcba9fc None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:09.883806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ebd1a-6b13-469c-b64f-54b58bcba9fc None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:09.884190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ebd1a-6b13-469c-b64f-54b58bcba9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:09.884351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc4f86f69ce745d79bd3d7ea5e9d061c', 'request_id': 'req-ca3cd395-94c8-4602-87b0-70bffd2d8d41'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.901918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc4f86f69ce745d79bd3d7ea5e9d061c', 'request_id': 'req-ca3cd395-94c8-4602-87b0-70bffd2d8d41'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.922056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-473ebd1a-6b13-469c-b64f-54b58bcba9fc None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:09.923193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc4f86f69ce745d79bd3d7ea5e9d061c', 'request_id': 'req-ca3cd395-94c8-4602-87b0-70bffd2d8d41'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.927653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-473ebd1a-6b13-469c-b64f-54b58bcba9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:09.931341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:09.940034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] The token cache is being invalidate because project dc4f86f69ce745d79bd3d7ea5e9d061c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:09.940324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.940979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-ca3cd395-94c8-4602-87b0-70bffd2d8d41 tempest-VolumeQuotaClassesTest-1217785258 tempest-VolumeQuotaClassesTest-1217785258-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:09.946707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1344/2667] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:09 2022] DELETE /identity/v3/projects/dc4f86f69ce745d79bd3d7ea5e9d061c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:10.061844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1324/2668] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:10.063696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21315a45-81c2-4800-8eef-3552b67ce9c5 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.064041 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21315a45-81c2-4800-8eef-3552b67ce9c5 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.064343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21315a45-81c2-4800-8eef-3552b67ce9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.085724 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-21315a45-81c2-4800-8eef-3552b67ce9c5 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:10.087089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-21315a45-81c2-4800-8eef-3552b67ce9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:10.105751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1345/2669] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:10.109914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ebd1a-6b13-469c-b64f-54b58bcba9fc None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:10.125281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ebd1a-6b13-469c-b64f-54b58bcba9fc None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:10.131672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce670182-97b9-4503-b653-8c8c792cf0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:10.132414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce670182-97b9-4503-b653-8c8c792cf0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.132738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce670182-97b9-4503-b653-8c8c792cf0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.133439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce670182-97b9-4503-b653-8c8c792cf0c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.133687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce670182-97b9-4503-b653-8c8c792cf0c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:02:10.134005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce670182-97b9-4503-b653-8c8c792cf0c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:10.135661 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce670182-97b9-4503-b653-8c8c792cf0c7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:10.151085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1325/2670] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:02:10.155242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21315a45-81c2-4800-8eef-3552b67ce9c5 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:10.167341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21315a45-81c2-4800-8eef-3552b67ce9c5 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:10.172663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888cf1c-1212-4428-8047-48d7fc540b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:10.173446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888cf1c-1212-4428-8047-48d7fc540b0c admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.173832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888cf1c-1212-4428-8047-48d7fc540b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.174359 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888cf1c-1212-4428-8047-48d7fc540b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.175311 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888cf1c-1212-4428-8047-48d7fc540b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:10.177247 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888cf1c-1212-4428-8047-48d7fc540b0c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:10.227206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1346/2671] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:10 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:10.231105 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce670182-97b9-4503-b653-8c8c792cf0c7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:10.242674 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce670182-97b9-4503-b653-8c8c792cf0c7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:10.280279 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:10.338516 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f35c3c-3d37-4f69-9b7a-26d40149b74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:10.339470 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f35c3c-3d37-4f69-9b7a-26d40149b74d admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.340024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f35c3c-3d37-4f69-9b7a-26d40149b74d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.340403 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f35c3c-3d37-4f69-9b7a-26d40149b74d admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.341357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f35c3c-3d37-4f69-9b7a-26d40149b74d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:10.344053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f35c3c-3d37-4f69-9b7a-26d40149b74d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:10.361448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888cf1c-1212-4428-8047-48d7fc540b0c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:10.364708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4888cf1c-1212-4428-8047-48d7fc540b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:10.388077 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888cf1c-1212-4428-8047-48d7fc540b0c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:10.388635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1326/2672] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:10 2022] POST /identity/v3/users => generated 284 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:10.392493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f35c3c-3d37-4f69-9b7a-26d40149b74d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:10.400701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:10.402036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.402419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.403469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] PATH_INFO: `/v3/users/9af273a454674517bcb4d8c3e043c094` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.405281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af273a454674517bcb4d8c3e043c094)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:10.407212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f35c3c-3d37-4f69-9b7a-26d40149b74d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:10.407714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:10.412151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e4fac-b2ac-4fc2-bd68-4b2de47dd922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:10.413806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e4fac-b2ac-4fc2-bd68-4b2de47dd922 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.414099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e4fac-b2ac-4fc2-bd68-4b2de47dd922 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.414385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e4fac-b2ac-4fc2-bd68-4b2de47dd922 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.415184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e4fac-b2ac-4fc2-bd68-4b2de47dd922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:10.416780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e4fac-b2ac-4fc2-bd68-4b2de47dd922 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:10.430251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1327/2673] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:10.438048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e4fac-b2ac-4fc2-bd68-4b2de47dd922 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:10.459849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e4fac-b2ac-4fc2-bd68-4b2de47dd922 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:10.459849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f26ea-9999-4f09-b6c7-ac458818588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:10.465550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f26ea-9999-4f09-b6c7-ac458818588d admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.465550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f26ea-9999-4f09-b6c7-ac458818588d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.465550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f26ea-9999-4f09-b6c7-ac458818588d admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.465550 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f26ea-9999-4f09-b6c7-ac458818588d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:10.468873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f26ea-9999-4f09-b6c7-ac458818588d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:10.490260 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1328/2674] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:10.497404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f26ea-9999-4f09-b6c7-ac458818588d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:10.513261 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f26ea-9999-4f09-b6c7-ac458818588d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:10.521846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7831f-d7ce-4244-a6ad-dd6cd3295ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:10.523180 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7831f-d7ce-4244-a6ad-dd6cd3295ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.523649 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7831f-d7ce-4244-a6ad-dd6cd3295ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.524227 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7831f-d7ce-4244-a6ad-dd6cd3295ccb admin admin] PATH_INFO: `/v3/projects/da0e896db70545aca84d82237371acbf/users/6a1ab09f4a374f2ea4dd26aa06357e89/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.531529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af273a454674517bcb4d8c3e043c094'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:10.550666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7831f-d7ce-4244-a6ad-dd6cd3295ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=da0e896db70545aca84d82237371acbf, user_id=6a1ab09f4a374f2ea4dd26aa06357e89, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:10.555024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7831f-d7ce-4244-a6ad-dd6cd3295ccb admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:10.555282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af273a454674517bcb4d8c3e043c094'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:10.565087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af273a454674517bcb4d8c3e043c094'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:10.565810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af273a454674517bcb4d8c3e043c094'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:10.583107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1329/2675] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:10 2022] PUT /identity/v3/projects/da0e896db70545aca84d82237371acbf/users/6a1ab09f4a374f2ea4dd26aa06357e89/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:10.592013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-4c544463-2b95-4c31-b407-6a2d43201c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af273a454674517bcb4d8c3e043c094'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:10.594764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e68c75-e0d5-4561-bb74-13a09ed2adc6 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.595249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e68c75-e0d5-4561-bb74-13a09ed2adc6 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.595706 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e68c75-e0d5-4561-bb74-13a09ed2adc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.599389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1347/2676] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:10 2022] DELETE /identity/v3/users/9af273a454674517bcb4d8c3e043c094 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:10.604703 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735ef05-41f8-4dba-a609-02962ec41d9d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:10.604999 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735ef05-41f8-4dba-a609-02962ec41d9d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:10.605281 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735ef05-41f8-4dba-a609-02962ec41d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:10.693387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-0735ef05-41f8-4dba-a609-02962ec41d9d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:10.699488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0735ef05-41f8-4dba-a609-02962ec41d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:10.728198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-27e68c75-e0d5-4561-bb74-13a09ed2adc6 None None] MFA Rules not processed for user `6a1ab09f4a374f2ea4dd26aa06357e89`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:10.736563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-27e68c75-e0d5-4561-bb74-13a09ed2adc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:10.871744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1348/2677] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:10.878215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735ef05-41f8-4dba-a609-02962ec41d9d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:10.881661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-0735ef05-41f8-4dba-a609-02962ec41d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:10.942341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1330/2678] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:10.953845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e68c75-e0d5-4561-bb74-13a09ed2adc6 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:10.957533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-27e68c75-e0d5-4561-bb74-13a09ed2adc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:10.978466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:11.015241 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735ef05-41f8-4dba-a609-02962ec41d9d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.020218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e68c75-e0d5-4561-bb74-13a09ed2adc6 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.021328 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915d51b-85bb-43de-b310-230182aa808d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.024343 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915d51b-85bb-43de-b310-230182aa808d service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.025529 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915d51b-85bb-43de-b310-230182aa808d service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.025781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915d51b-85bb-43de-b310-230182aa808d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.032499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b808cc-4d1d-42dc-83f5-955db42b22a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.032499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b808cc-4d1d-42dc-83f5-955db42b22a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.032499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b808cc-4d1d-42dc-83f5-955db42b22a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.032499 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b808cc-4d1d-42dc-83f5-955db42b22a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.033208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b808cc-4d1d-42dc-83f5-955db42b22a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.035148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b808cc-4d1d-42dc-83f5-955db42b22a2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.046093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915d51b-85bb-43de-b310-230182aa808d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.055804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915d51b-85bb-43de-b310-230182aa808d service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.080144 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1349/2679] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:02:11.098273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1331/2680] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:10 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:11.099491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915d51b-85bb-43de-b310-230182aa808d service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:11.112907 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915d51b-85bb-43de-b310-230182aa808d service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.244667 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69148a99-1cf5-4a7e-9678-ab3e210cf438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.245895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69148a99-1cf5-4a7e-9678-ab3e210cf438 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.245895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69148a99-1cf5-4a7e-9678-ab3e210cf438 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.245895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69148a99-1cf5-4a7e-9678-ab3e210cf438 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.246598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69148a99-1cf5-4a7e-9678-ab3e210cf438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.249043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69148a99-1cf5-4a7e-9678-ab3e210cf438 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.303535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1350/2681] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:11 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:02:11.304865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b808cc-4d1d-42dc-83f5-955db42b22a2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:11.316397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b808cc-4d1d-42dc-83f5-955db42b22a2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.325908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8de60-079d-469f-8929-4f0ce1c53adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.327515 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8de60-079d-469f-8929-4f0ce1c53adc admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.328126 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8de60-079d-469f-8929-4f0ce1c53adc admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.328564 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8de60-079d-469f-8929-4f0ce1c53adc admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.329517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8de60-079d-469f-8929-4f0ce1c53adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.332526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8de60-079d-469f-8929-4f0ce1c53adc admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.351897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1332/2682] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:11.352649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69148a99-1cf5-4a7e-9678-ab3e210cf438 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:11.372552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69148a99-1cf5-4a7e-9678-ab3e210cf438 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.381240 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3612b9f-b1a3-41a6-86ac-dccad2e85194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.383159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3612b9f-b1a3-41a6-86ac-dccad2e85194 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.385906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3612b9f-b1a3-41a6-86ac-dccad2e85194 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.386466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3612b9f-b1a3-41a6-86ac-dccad2e85194 admin admin] PATH_INFO: `/v3/projects/d4d6ad5f123b4de093bdb325bd3f92dc/users/add9d607265347f6b60c1034977e5ad2/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.431718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3612b9f-b1a3-41a6-86ac-dccad2e85194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d6ad5f123b4de093bdb325bd3f92dc, user_id=add9d607265347f6b60c1034977e5ad2, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.434840 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3612b9f-b1a3-41a6-86ac-dccad2e85194 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.509806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1351/2683] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:11 2022] PUT /identity/v3/projects/d4d6ad5f123b4de093bdb325bd3f92dc/users/add9d607265347f6b60c1034977e5ad2/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:11.513880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8de60-079d-469f-8929-4f0ce1c53adc admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:11.531880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8de60-079d-469f-8929-4f0ce1c53adc admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.548992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3612b9f-b1a3-41a6-86ac-dccad2e85194 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:11.552958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c3612b9f-b1a3-41a6-86ac-dccad2e85194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:11.615725 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:11.680130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c08b9f-446c-469e-ad22-3a8e92db358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.680822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c08b9f-446c-469e-ad22-3a8e92db358d admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.681109 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c08b9f-446c-469e-ad22-3a8e92db358d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.681389 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c08b9f-446c-469e-ad22-3a8e92db358d admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.682172 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c08b9f-446c-469e-ad22-3a8e92db358d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.683798 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c08b9f-446c-469e-ad22-3a8e92db358d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.695193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3612b9f-b1a3-41a6-86ac-dccad2e85194 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.701716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b652d24-3edd-4ac6-b86d-a939a8e7de6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.702528 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b652d24-3edd-4ac6-b86d-a939a8e7de6a service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.702983 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b652d24-3edd-4ac6-b86d-a939a8e7de6a service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.703353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b652d24-3edd-4ac6-b86d-a939a8e7de6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.703838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1333/2684] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:11.704741 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c08b9f-446c-469e-ad22-3a8e92db358d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:11.707049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-15c08b9f-446c-469e-ad22-3a8e92db358d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:11.710069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-4b652d24-3edd-4ac6-b86d-a939a8e7de6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:11.723075 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c08b9f-446c-469e-ad22-3a8e92db358d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.727044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b652d24-3edd-4ac6-b86d-a939a8e7de6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.729453 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b652d24-3edd-4ac6-b86d-a939a8e7de6a service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.731540 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.733776 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.734237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.734825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] PATH_INFO: `/v3/projects/ee07a0779fa943ba93c48c46ede1ca25` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.736532 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee07a0779fa943ba93c48c46ede1ca25)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.738852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.750369 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1352/2685] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:02:11.751324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b652d24-3edd-4ac6-b86d-a939a8e7de6a service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:11.763517 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b652d24-3edd-4ac6-b86d-a939a8e7de6a service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.773819 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9aed7d-3ee6-47e3-aa7a-0de05bd92919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.776505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aed7d-3ee6-47e3-aa7a-0de05bd92919 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.776807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aed7d-3ee6-47e3-aa7a-0de05bd92919 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.777096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aed7d-3ee6-47e3-aa7a-0de05bd92919 admin admin] PATH_INFO: `/v3/projects/d4d6ad5f123b4de093bdb325bd3f92dc/users/add9d607265347f6b60c1034977e5ad2/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.780074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9aed7d-3ee6-47e3-aa7a-0de05bd92919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d6ad5f123b4de093bdb325bd3f92dc, user_id=add9d607265347f6b60c1034977e5ad2, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.784028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9aed7d-3ee6-47e3-aa7a-0de05bd92919 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.800746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1353/2686] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:11 2022] PUT /identity/v3/projects/d4d6ad5f123b4de093bdb325bd3f92dc/users/add9d607265347f6b60c1034977e5ad2/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:11.807663 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee07a0779fa943ba93c48c46ede1ca25', 'request_id': 'req-d096b40c-ed45-4e45-b933-20e484651693'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:11.812931 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f199501-9b8b-4fef-9c0a-bdaac6b25874 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.813418 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f199501-9b8b-4fef-9c0a-bdaac6b25874 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.813863 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f199501-9b8b-4fef-9c0a-bdaac6b25874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.814476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee07a0779fa943ba93c48c46ede1ca25', 'request_id': 'req-d096b40c-ed45-4e45-b933-20e484651693'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:11.823331 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee07a0779fa943ba93c48c46ede1ca25', 'request_id': 'req-d096b40c-ed45-4e45-b933-20e484651693'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:11.838596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] The token cache is being invalidate because project ee07a0779fa943ba93c48c46ede1ca25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:11.838919 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:11.839874 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:11.841854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1334/2687] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:11 2022] DELETE /identity/v3/projects/ee07a0779fa943ba93c48c46ede1ca25 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:11.846026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:11.846637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9f199501-9b8b-4fef-9c0a-bdaac6b25874 None None] MFA Rules not processed for user `add9d607265347f6b60c1034977e5ad2`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:11.847905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9f199501-9b8b-4fef-9c0a-bdaac6b25874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:11.848358 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:11.857049 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:11.918576 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096b40c-ed45-4e45-b933-20e484651693 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.919651 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1354/2688] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:11 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:11.931836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f199501-9b8b-4fef-9c0a-bdaac6b25874 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:11.933285 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9f199501-9b8b-4fef-9c0a-bdaac6b25874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:11.971087 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:11.979053 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.980595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.981401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.982208 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] PATH_INFO: `/v3/users/7a5f6d1f04b0408090acc5e3ca72bc78` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:11.984235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5f6d1f04b0408090acc5e3ca72bc78)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:11.986428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:11.992087 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f199501-9b8b-4fef-9c0a-bdaac6b25874 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:11.995990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf329c31-bdf6-421a-a694-a90b9d143509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:11.996629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf329c31-bdf6-421a-a694-a90b9d143509 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:11.996911 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf329c31-bdf6-421a-a694-a90b9d143509 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:11.997183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf329c31-bdf6-421a-a694-a90b9d143509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:12.007630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf329c31-bdf6-421a-a694-a90b9d143509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:12.009208 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf329c31-bdf6-421a-a694-a90b9d143509 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:12.019805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1355/2689] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:11 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:02:12.073752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5f6d1f04b0408090acc5e3ca72bc78'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.096457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5f6d1f04b0408090acc5e3ca72bc78'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.105320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5f6d1f04b0408090acc5e3ca72bc78'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.105320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5f6d1f04b0408090acc5e3ca72bc78'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.106050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-4b5ded84-28c7-48fa-b801-7aee3442706d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5f6d1f04b0408090acc5e3ca72bc78'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.126303 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1335/2690] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:11 2022] DELETE /identity/v3/users/7a5f6d1f04b0408090acc5e3ca72bc78 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:12.444762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf329c31-bdf6-421a-a694-a90b9d143509 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:12.464292 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf329c31-bdf6-421a-a694-a90b9d143509 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:12.536403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:12.537259 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:12.537805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:12.538105 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] PATH_INFO: `/v3/projects/d1461bf3e65b440492e4bfb08a781c01` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:12.539422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1461bf3e65b440492e4bfb08a781c01)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:12.541317 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:12.593806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1461bf3e65b440492e4bfb08a781c01', 'request_id': 'req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.595802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:12.601146 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1461bf3e65b440492e4bfb08a781c01', 'request_id': 'req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.610263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1461bf3e65b440492e4bfb08a781c01', 'request_id': 'req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.622054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] The token cache is being invalidate because project d1461bf3e65b440492e4bfb08a781c01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:12.622351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.623061 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6dc5d8cf-bc7e-47eb-902b-f49ec77457bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:12.624602 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1356/2691] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:12 2022] DELETE /identity/v3/projects/d1461bf3e65b440492e4bfb08a781c01 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:12.629410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ce399-d38d-45f7-be3e-a1af867a31ba None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:12.629866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ce399-d38d-45f7-be3e-a1af867a31ba None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:12.629979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ce399-d38d-45f7-be3e-a1af867a31ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:12.650923 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-b61ce399-d38d-45f7-be3e-a1af867a31ba None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:12.651397 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-b61ce399-d38d-45f7-be3e-a1af867a31ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:12.716799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1336/2692] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:12.724507 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de9e20-c5fb-447d-9db9-14a0d7acd11d None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:12.724895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de9e20-c5fb-447d-9db9-14a0d7acd11d None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:12.725314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de9e20-c5fb-447d-9db9-14a0d7acd11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:12.750030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-b1de9e20-c5fb-447d-9db9-14a0d7acd11d None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:12.751434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-b1de9e20-c5fb-447d-9db9-14a0d7acd11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:12.763243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1357/2693] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:12.767192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ce399-d38d-45f7-be3e-a1af867a31ba None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:12.778150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ce399-d38d-45f7-be3e-a1af867a31ba None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:12.782809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857e146-877f-4ce1-83ba-ebc491feadee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:12.783468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857e146-877f-4ce1-83ba-ebc491feadee admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:12.783752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857e146-877f-4ce1-83ba-ebc491feadee admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:12.784032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857e146-877f-4ce1-83ba-ebc491feadee admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:12.784755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857e146-877f-4ce1-83ba-ebc491feadee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:02:12.785068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857e146-877f-4ce1-83ba-ebc491feadee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:12.786643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857e146-877f-4ce1-83ba-ebc491feadee admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:12.798880 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1337/2694] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:02:12.803173 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de9e20-c5fb-447d-9db9-14a0d7acd11d None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:12.813396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de9e20-c5fb-447d-9db9-14a0d7acd11d None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:12.818211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee713a-a71f-4ead-bb57-d00cfb7ce5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:12.818921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee713a-a71f-4ead-bb57-d00cfb7ce5fb admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:12.819204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee713a-a71f-4ead-bb57-d00cfb7ce5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:12.820119 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee713a-a71f-4ead-bb57-d00cfb7ce5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:12.820820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee713a-a71f-4ead-bb57-d00cfb7ce5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:12.822720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee713a-a71f-4ead-bb57-d00cfb7ce5fb admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:12.858758 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1358/2695] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:12 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:12.860332 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857e146-877f-4ce1-83ba-ebc491feadee admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:12.871460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857e146-877f-4ce1-83ba-ebc491feadee admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:12.938530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847d445-45eb-490f-9ab3-124ad671e649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:12.939433 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d445-45eb-490f-9ab3-124ad671e649 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:12.939881 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d445-45eb-490f-9ab3-124ad671e649 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:12.940032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d445-45eb-490f-9ab3-124ad671e649 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:12.941310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847d445-45eb-490f-9ab3-124ad671e649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:12.943266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847d445-45eb-490f-9ab3-124ad671e649 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:12.953232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:12.967345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1338/2696] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:12 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:12.971476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee713a-a71f-4ead-bb57-d00cfb7ce5fb admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:12.982039 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee713a-a71f-4ead-bb57-d00cfb7ce5fb admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:12.989670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff7710-e128-406e-aa80-ca1b9bb2b188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:12.990335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff7710-e128-406e-aa80-ca1b9bb2b188 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:12.990614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff7710-e128-406e-aa80-ca1b9bb2b188 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:12.990927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff7710-e128-406e-aa80-ca1b9bb2b188 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:12.992199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff7710-e128-406e-aa80-ca1b9bb2b188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:12.993906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff7710-e128-406e-aa80-ca1b9bb2b188 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.006791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1359/2697] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:13.010372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847d445-45eb-490f-9ab3-124ad671e649 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.020651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847d445-45eb-490f-9ab3-124ad671e649 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.029359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df27601-7daf-4976-bde4-b596fdeb3ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.029359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df27601-7daf-4976-bde4-b596fdeb3ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.029359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df27601-7daf-4976-bde4-b596fdeb3ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.029359 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df27601-7daf-4976-bde4-b596fdeb3ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.029754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df27601-7daf-4976-bde4-b596fdeb3ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.029754 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df27601-7daf-4976-bde4-b596fdeb3ac4 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.040276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1339/2698] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:13.044318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff7710-e128-406e-aa80-ca1b9bb2b188 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.055151 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff7710-e128-406e-aa80-ca1b9bb2b188 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.059820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca22ee0-a654-43a5-a875-a879e6d21b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.060733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca22ee0-a654-43a5-a875-a879e6d21b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.061018 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca22ee0-a654-43a5-a875-a879e6d21b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.061351 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca22ee0-a654-43a5-a875-a879e6d21b25 admin admin] PATH_INFO: `/v3/projects/36dc9ef6806e48ec9081a3f58178859d/users/018b884bbd1e4d4ba0c857ff053669a1/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.076806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca22ee0-a654-43a5-a875-a879e6d21b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36dc9ef6806e48ec9081a3f58178859d, user_id=018b884bbd1e4d4ba0c857ff053669a1, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.079492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca22ee0-a654-43a5-a875-a879e6d21b25 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.095811 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1360/2699] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:13 2022] PUT /identity/v3/projects/36dc9ef6806e48ec9081a3f58178859d/users/018b884bbd1e4d4ba0c857ff053669a1/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:13.107150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557ebe6-d4ce-4c85-8478-873bf552f52f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.107150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557ebe6-d4ce-4c85-8478-873bf552f52f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.107150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557ebe6-d4ce-4c85-8478-873bf552f52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.135430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8557ebe6-d4ce-4c85-8478-873bf552f52f None None] MFA Rules not processed for user `018b884bbd1e4d4ba0c857ff053669a1`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:13.136646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8557ebe6-d4ce-4c85-8478-873bf552f52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:13.206495 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1340/2700] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:13.210759 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca22ee0-a654-43a5-a875-a879e6d21b25 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.283559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca22ee0-a654-43a5-a875-a879e6d21b25 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.289327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4093657-3e39-451f-9a2a-c10706bc3c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.289995 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4093657-3e39-451f-9a2a-c10706bc3c1a admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.290455 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4093657-3e39-451f-9a2a-c10706bc3c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.290808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4093657-3e39-451f-9a2a-c10706bc3c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.294685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4093657-3e39-451f-9a2a-c10706bc3c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.296676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4093657-3e39-451f-9a2a-c10706bc3c1a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.329565 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1361/2701] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:13 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:13.333813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557ebe6-d4ce-4c85-8478-873bf552f52f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.352117 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557ebe6-d4ce-4c85-8478-873bf552f52f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.434846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bce5b-a745-439e-9d22-8cd1d713492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.435648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bce5b-a745-439e-9d22-8cd1d713492b admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.436035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bce5b-a745-439e-9d22-8cd1d713492b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.436449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bce5b-a745-439e-9d22-8cd1d713492b admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.437215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bce5b-a745-439e-9d22-8cd1d713492b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.439777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bce5b-a745-439e-9d22-8cd1d713492b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.464282 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4093657-3e39-451f-9a2a-c10706bc3c1a admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.466416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a4093657-3e39-451f-9a2a-c10706bc3c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:13.471660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1341/2702] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:13 2022] POST /identity/v3/users => generated 301 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:02:13.475402 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bce5b-a745-439e-9d22-8cd1d713492b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.485887 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4093657-3e39-451f-9a2a-c10706bc3c1a admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.490947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.492400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.492681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.492959 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] PATH_INFO: `/v3/users/6a1ab09f4a374f2ea4dd26aa06357e89` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.493904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bce5b-a745-439e-9d22-8cd1d713492b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.494183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a1ab09f4a374f2ea4dd26aa06357e89)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.496185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.498792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19673866-44cb-4e6f-9267-8e649f7ebe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.499469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19673866-44cb-4e6f-9267-8e649f7ebe9e admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.499756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19673866-44cb-4e6f-9267-8e649f7ebe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.500189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19673866-44cb-4e6f-9267-8e649f7ebe9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.501119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19673866-44cb-4e6f-9267-8e649f7ebe9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.502718 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19673866-44cb-4e6f-9267-8e649f7ebe9e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.514887 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1342/2703] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:13.517400 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19673866-44cb-4e6f-9267-8e649f7ebe9e admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.527108 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19673866-44cb-4e6f-9267-8e649f7ebe9e admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.532650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce67a51-21d9-4189-98a9-f519e1e4becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.533528 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce67a51-21d9-4189-98a9-f519e1e4becc admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.533818 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce67a51-21d9-4189-98a9-f519e1e4becc admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.534110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce67a51-21d9-4189-98a9-f519e1e4becc admin admin] PATH_INFO: `/v3/projects/2e84d851c41e4adda41a5c3d7e34b241/users/fce187b760a24de8aa0fa67325297703/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.550543 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce67a51-21d9-4189-98a9-f519e1e4becc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e84d851c41e4adda41a5c3d7e34b241, user_id=fce187b760a24de8aa0fa67325297703, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.552335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a1ab09f4a374f2ea4dd26aa06357e89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:13.553248 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce67a51-21d9-4189-98a9-f519e1e4becc admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.561474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a1ab09f4a374f2ea4dd26aa06357e89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:13.563047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=84,keystone:INSERT=5,keystone:DELETE=11 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:13.568861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1343/2704] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:13 2022] PUT /identity/v3/projects/2e84d851c41e4adda41a5c3d7e34b241/users/fce187b760a24de8aa0fa67325297703/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:13.572841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a1ab09f4a374f2ea4dd26aa06357e89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:13.573084 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce67a51-21d9-4189-98a9-f519e1e4becc admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.573946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a1ab09f4a374f2ea4dd26aa06357e89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:13.587378 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7965513e-aaa4-42f9-b280-22dead2d3e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a1ab09f4a374f2ea4dd26aa06357e89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:13.593804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1362/2705] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:13 2022] DELETE /identity/v3/users/6a1ab09f4a374f2ea4dd26aa06357e89 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:13.597788 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fa235-ce64-4733-aaba-6cd031389641 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.598360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fa235-ce64-4733-aaba-6cd031389641 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.598360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fa235-ce64-4733-aaba-6cd031389641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.604870 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:13.615874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ca7fa235-ce64-4733-aaba-6cd031389641 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:13.617718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ca7fa235-ce64-4733-aaba-6cd031389641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:13.639132 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce67a51-21d9-4189-98a9-f519e1e4becc admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.690396 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1363/2706] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:13.697760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ce0a-82d5-4687-a00e-72efccba6882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.698451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ce0a-82d5-4687-a00e-72efccba6882 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.698687 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ce0a-82d5-4687-a00e-72efccba6882 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.699116 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ce0a-82d5-4687-a00e-72efccba6882 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.699723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ce0a-82d5-4687-a00e-72efccba6882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.700028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fa235-ce64-4733-aaba-6cd031389641 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.701317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ce0a-82d5-4687-a00e-72efccba6882 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.701524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ca7fa235-ce64-4733-aaba-6cd031389641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:13.712443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1344/2707] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:13.716151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ce0a-82d5-4687-a00e-72efccba6882 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.717843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-aba2ce0a-82d5-4687-a00e-72efccba6882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:13.731068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ce0a-82d5-4687-a00e-72efccba6882 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.737444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c8748-9518-4543-bfc8-6a06b5039218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.738342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c8748-9518-4543-bfc8-6a06b5039218 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.738629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c8748-9518-4543-bfc8-6a06b5039218 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.738946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c8748-9518-4543-bfc8-6a06b5039218 admin admin] PATH_INFO: `/v3/projects/2e84d851c41e4adda41a5c3d7e34b241/users/fce187b760a24de8aa0fa67325297703/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.740922 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c8748-9518-4543-bfc8-6a06b5039218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e84d851c41e4adda41a5c3d7e34b241, user_id=fce187b760a24de8aa0fa67325297703, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.743650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c8748-9518-4543-bfc8-6a06b5039218 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.760194 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1345/2708] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:13 2022] PUT /identity/v3/projects/2e84d851c41e4adda41a5c3d7e34b241/users/fce187b760a24de8aa0fa67325297703/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:13.765681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4f613-9189-4694-8849-d48e4553ead7 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.765964 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4f613-9189-4694-8849-d48e4553ead7 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.766247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4f613-9189-4694-8849-d48e4553ead7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.770970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fa235-ce64-4733-aaba-6cd031389641 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.795936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-16a4f613-9189-4694-8849-d48e4553ead7 None None] MFA Rules not processed for user `fce187b760a24de8aa0fa67325297703`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:13.797141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-16a4f613-9189-4694-8849-d48e4553ead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:13.823871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3de22-5eed-4dc9-b355-07832f70366a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.824753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3de22-5eed-4dc9-b355-07832f70366a service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.825037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3de22-5eed-4dc9-b355-07832f70366a service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.825364 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3de22-5eed-4dc9-b355-07832f70366a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.864297 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1346/2709] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:13.869644 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:13.875408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4f613-9189-4694-8849-d48e4553ead7 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:13.876917 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-16a4f613-9189-4694-8849-d48e4553ead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:13.889495 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3de22-5eed-4dc9-b355-07832f70366a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.890960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3de22-5eed-4dc9-b355-07832f70366a service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.901045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1364/2710] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:02:13.923052 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:13.938799 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4f613-9189-4694-8849-d48e4553ead7 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:13.944765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88398c9-11d2-4404-9682-e9024106bfbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:13.945804 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88398c9-11d2-4404-9682-e9024106bfbc service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:13.946270 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88398c9-11d2-4404-9682-e9024106bfbc service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:13.946738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88398c9-11d2-4404-9682-e9024106bfbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:13.957156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88398c9-11d2-4404-9682-e9024106bfbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:13.958514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88398c9-11d2-4404-9682-e9024106bfbc service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:13.969672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1347/2711] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:14.103976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3de22-5eed-4dc9-b355-07832f70366a service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:14.113699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3de22-5eed-4dc9-b355-07832f70366a service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:14.118592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:14.119797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:14.120078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:14.120355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] PATH_INFO: `/v3/users/018b884bbd1e4d4ba0c857ff053669a1` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:14.121543 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018b884bbd1e4d4ba0c857ff053669a1)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:14.123587 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:14.173832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018b884bbd1e4d4ba0c857ff053669a1'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.182192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88398c9-11d2-4404-9682-e9024106bfbc service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:14.183839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e88398c9-11d2-4404-9682-e9024106bfbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:14.184769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018b884bbd1e4d4ba0c857ff053669a1'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.197891 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018b884bbd1e4d4ba0c857ff053669a1'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.198603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018b884bbd1e4d4ba0c857ff053669a1'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.199382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88398c9-11d2-4404-9682-e9024106bfbc service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:14.204345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:14.205449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:14.205849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:14.206262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] PATH_INFO: `/v3/projects/da0e896db70545aca84d82237371acbf` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:14.207651 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da0e896db70545aca84d82237371acbf)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:14.209845 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:14.213688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-e8a9ffe0-2c58-452e-b4e8-34737d0b8964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018b884bbd1e4d4ba0c857ff053669a1'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.220243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1365/2712] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:14 2022] DELETE /identity/v3/users/018b884bbd1e4d4ba0c857ff053669a1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:14.224685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982483b2-cdfe-41e9-aa83-b7c3893b5359 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:14.224980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982483b2-cdfe-41e9-aa83-b7c3893b5359 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:14.225263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982483b2-cdfe-41e9-aa83-b7c3893b5359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:14.244670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-982483b2-cdfe-41e9-aa83-b7c3893b5359 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:14.245899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-982483b2-cdfe-41e9-aa83-b7c3893b5359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:14.260567 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da0e896db70545aca84d82237371acbf', 'request_id': 'req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.267001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da0e896db70545aca84d82237371acbf', 'request_id': 'req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.276166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da0e896db70545aca84d82237371acbf', 'request_id': 'req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.288542 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] The token cache is being invalidate because project da0e896db70545aca84d82237371acbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:14.288735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.289327 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.290894 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1348/2713] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:14 2022] DELETE /identity/v3/projects/da0e896db70545aca84d82237371acbf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:14.294583 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:14.295927 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:14.308016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1366/2714] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:14.317505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982483b2-cdfe-41e9-aa83-b7c3893b5359 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:14.318888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-982483b2-cdfe-41e9-aa83-b7c3893b5359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:14.365197 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0a77-9bec-4f1d-8366-d00ae8b02b50 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:14.370653 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:14.371612 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:14.371892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:14.372294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] PATH_INFO: `/v3/users/add9d607265347f6b60c1034977e5ad2` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:14.373529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=add9d607265347f6b60c1034977e5ad2)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:14.375493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:14.378842 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982483b2-cdfe-41e9-aa83-b7c3893b5359 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:14.382827 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128f9d7-43a6-4e8f-85fa-19efca81652c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:14.383523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128f9d7-43a6-4e8f-85fa-19efca81652c service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:14.383839 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128f9d7-43a6-4e8f-85fa-19efca81652c service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:14.384123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128f9d7-43a6-4e8f-85fa-19efca81652c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:14.395111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:14.400571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128f9d7-43a6-4e8f-85fa-19efca81652c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:14.402322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128f9d7-43a6-4e8f-85fa-19efca81652c service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:14.415475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1367/2715] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:02:14.434534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add9d607265347f6b60c1034977e5ad2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.446512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add9d607265347f6b60c1034977e5ad2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.456161 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add9d607265347f6b60c1034977e5ad2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.456931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add9d607265347f6b60c1034977e5ad2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.457673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add9d607265347f6b60c1034977e5ad2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.464860 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:14.484428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1349/2716] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:14 2022] DELETE /identity/v3/users/add9d607265347f6b60c1034977e5ad2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:14.841031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128f9d7-43a6-4e8f-85fa-19efca81652c service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:14.851050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128f9d7-43a6-4e8f-85fa-19efca81652c service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:14.890316 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:14.892008 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:14.919553 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:14.920459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:14.920768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:14.920959 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] PATH_INFO: `/v3/projects/d4d6ad5f123b4de093bdb325bd3f92dc` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:14.922175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d6ad5f123b4de093bdb325bd3f92dc)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:14.924521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:14.980325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe363de3-bbd4-404c-8cda-410712f5b93a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:14.984928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d6ad5f123b4de093bdb325bd3f92dc', 'request_id': 'req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:14.991279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d6ad5f123b4de093bdb325bd3f92dc', 'request_id': 'req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.000856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d6ad5f123b4de093bdb325bd3f92dc', 'request_id': 'req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.017878 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] The token cache is being invalidate because project d4d6ad5f123b4de093bdb325bd3f92dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:15.018634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.019096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-b6f85de5-229e-4e8e-aaf2-8db5498bcb81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.020894 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1368/2717] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:14 2022] DELETE /identity/v3/projects/d4d6ad5f123b4de093bdb325bd3f92dc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:15.026017 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9d2a3-868e-4137-92f7-e08e31e14b59 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.026324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9d2a3-868e-4137-92f7-e08e31e14b59 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.026598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9d2a3-868e-4137-92f7-e08e31e14b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.044668 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-d8c9d2a3-868e-4137-92f7-e08e31e14b59 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:15.045866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d8c9d2a3-868e-4137-92f7-e08e31e14b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:15.050041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.051047 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.051404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.051747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] PATH_INFO: `/v3/projects/36dc9ef6806e48ec9081a3f58178859d` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.053027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] RBAC: Authorizing `identity:delete_project(project_id=36dc9ef6806e48ec9081a3f58178859d)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.053882 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1369/2718] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:15.055212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.061332 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85a8e2-9bca-473b-957d-a3ee1c8aec8c None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.061614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85a8e2-9bca-473b-957d-a3ee1c8aec8c None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.061890 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85a8e2-9bca-473b-957d-a3ee1c8aec8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.080564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-fb85a8e2-9bca-473b-957d-a3ee1c8aec8c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:15.081874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fb85a8e2-9bca-473b-957d-a3ee1c8aec8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:15.090998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1370/2719] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:15.095342 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85a8e2-9bca-473b-957d-a3ee1c8aec8c None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.104233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85a8e2-9bca-473b-957d-a3ee1c8aec8c None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:15.106451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36dc9ef6806e48ec9081a3f58178859d', 'request_id': 'req-0522183e-ea26-439a-8832-ee1ec36ce47c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.112756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36dc9ef6806e48ec9081a3f58178859d', 'request_id': 'req-0522183e-ea26-439a-8832-ee1ec36ce47c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.121181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36dc9ef6806e48ec9081a3f58178859d', 'request_id': 'req-0522183e-ea26-439a-8832-ee1ec36ce47c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.145150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] The token cache is being invalidate because project 36dc9ef6806e48ec9081a3f58178859d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:15.145443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.146067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.147650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1350/2720] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:14 2022] DELETE /identity/v3/projects/36dc9ef6806e48ec9081a3f58178859d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:15.151570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.152892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:15.162344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.163424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.163708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.163989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.164721 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:02:15.165031 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.166631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.171990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:15.176593 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522183e-ea26-439a-8832-ee1ec36ce47c admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:15.182713 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1371/2721] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:02:15.182943 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.183605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.183898 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.184015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.184526 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] PATH_INFO: `/v3/users/fce187b760a24de8aa0fa67325297703` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.185810 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:15.186426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce187b760a24de8aa0fa67325297703)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.188641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.196032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:15.200048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc733e40-0d00-4a8d-85a4-30bd47d80008 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:15.206631 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf8f4c-4d55-453e-b6ce-eb76f6719c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.207755 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf8f4c-4d55-453e-b6ce-eb76f6719c28 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.208250 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf8f4c-4d55-453e-b6ce-eb76f6719c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.208715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf8f4c-4d55-453e-b6ce-eb76f6719c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.209576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf8f4c-4d55-453e-b6ce-eb76f6719c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.211535 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf8f4c-4d55-453e-b6ce-eb76f6719c28 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.242886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce187b760a24de8aa0fa67325297703'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.243133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1372/2722] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:15 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:02:15.243904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf8f4c-4d55-453e-b6ce-eb76f6719c28 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.251610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce187b760a24de8aa0fa67325297703'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.262498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce187b760a24de8aa0fa67325297703'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.263679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce187b760a24de8aa0fa67325297703'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.264226 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce187b760a24de8aa0fa67325297703'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.283365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1351/2723] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:15 2022] DELETE /identity/v3/users/fce187b760a24de8aa0fa67325297703 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:15.312188 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf8f4c-4d55-453e-b6ce-eb76f6719c28 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:15.371933 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773c20e-3ed8-46d1-98de-f1f257630ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.372636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773c20e-3ed8-46d1-98de-f1f257630ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.373067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773c20e-3ed8-46d1-98de-f1f257630ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.373355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773c20e-3ed8-46d1-98de-f1f257630ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.374157 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773c20e-3ed8-46d1-98de-f1f257630ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.376123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773c20e-3ed8-46d1-98de-f1f257630ad5 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.399949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1373/2724] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:15 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:02:15.404218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.406055 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:15.420601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2efe3-10f4-41d7-a039-a0512b4a8bcc admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:15.425519 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcb172-57bc-472b-9d3a-f8635c8f464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.426186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb172-57bc-472b-9d3a-f8635c8f464a admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.426471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb172-57bc-472b-9d3a-f8635c8f464a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.426912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb172-57bc-472b-9d3a-f8635c8f464a admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.428266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcb172-57bc-472b-9d3a-f8635c8f464a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.430408 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcb172-57bc-472b-9d3a-f8635c8f464a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.446640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1352/2725] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:15.448026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773c20e-3ed8-46d1-98de-f1f257630ad5 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.475367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773c20e-3ed8-46d1-98de-f1f257630ad5 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:15.482937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40679bdc-b762-489a-9804-43134b2653da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.483904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40679bdc-b762-489a-9804-43134b2653da admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.484356 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40679bdc-b762-489a-9804-43134b2653da admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.485043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40679bdc-b762-489a-9804-43134b2653da admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.485539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40679bdc-b762-489a-9804-43134b2653da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.487263 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40679bdc-b762-489a-9804-43134b2653da admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.499476 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1374/2726] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:15.503672 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcb172-57bc-472b-9d3a-f8635c8f464a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.516690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcb172-57bc-472b-9d3a-f8635c8f464a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:15.525982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4a8bf-2666-41b9-9e46-6d61218e1f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.526965 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4a8bf-2666-41b9-9e46-6d61218e1f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.527254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4a8bf-2666-41b9-9e46-6d61218e1f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.527534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4a8bf-2666-41b9-9e46-6d61218e1f2a admin admin] PATH_INFO: `/v3/projects/fe7d80ef8e1643fe98e7b0d9c2222f26/users/dedef0d8de0142149f72a46371099af5/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.546346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4a8bf-2666-41b9-9e46-6d61218e1f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe7d80ef8e1643fe98e7b0d9c2222f26, user_id=dedef0d8de0142149f72a46371099af5, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.549811 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4a8bf-2666-41b9-9e46-6d61218e1f2a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.567745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1353/2727] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:15 2022] PUT /identity/v3/projects/fe7d80ef8e1643fe98e7b0d9c2222f26/users/dedef0d8de0142149f72a46371099af5/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:15.587219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac38b83-72a3-4f16-932c-dc88a84d2474 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.587743 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac38b83-72a3-4f16-932c-dc88a84d2474 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.588387 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac38b83-72a3-4f16-932c-dc88a84d2474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.639925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-aac38b83-72a3-4f16-932c-dc88a84d2474 None None] MFA Rules not processed for user `dedef0d8de0142149f72a46371099af5`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:15.641510 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-aac38b83-72a3-4f16-932c-dc88a84d2474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:15.646778 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4a8bf-2666-41b9-9e46-6d61218e1f2a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.648782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fca4a8bf-2666-41b9-9e46-6d61218e1f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:15.737915 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:15.746037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1375/2728] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:15.750914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac38b83-72a3-4f16-932c-dc88a84d2474 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.761212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4a8bf-2666-41b9-9e46-6d61218e1f2a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:15.763104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac38b83-72a3-4f16-932c-dc88a84d2474 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:15.765724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.766579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.766907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.767173 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] PATH_INFO: `/v3/projects/2e84d851c41e4adda41a5c3d7e34b241` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.768096 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc14733-0e1a-433b-8edc-fcd3ba81d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:15.768338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e84d851c41e4adda41a5c3d7e34b241)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.768739 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc14733-0e1a-433b-8edc-fcd3ba81d143 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.769022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc14733-0e1a-433b-8edc-fcd3ba81d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.769302 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc14733-0e1a-433b-8edc-fcd3ba81d143 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.769981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc14733-0e1a-433b-8edc-fcd3ba81d143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:15.770240 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.771974 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc14733-0e1a-433b-8edc-fcd3ba81d143 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:15.805688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1376/2729] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:15 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:02:15.806416 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc14733-0e1a-433b-8edc-fcd3ba81d143 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:15.824852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:15.828534 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e84d851c41e4adda41a5c3d7e34b241', 'request_id': 'req-c956fdb3-af99-497f-96bc-8b2e955cb0ff'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.837563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e84d851c41e4adda41a5c3d7e34b241', 'request_id': 'req-c956fdb3-af99-497f-96bc-8b2e955cb0ff'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.860377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e84d851c41e4adda41a5c3d7e34b241', 'request_id': 'req-c956fdb3-af99-497f-96bc-8b2e955cb0ff'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.873941 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] The token cache is being invalidate because project 2e84d851c41e4adda41a5c3d7e34b241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:15.874794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.875605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c956fdb3-af99-497f-96bc-8b2e955cb0ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:15.877288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1354/2730] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:15 2022] DELETE /identity/v3/projects/2e84d851c41e4adda41a5c3d7e34b241 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:15.882544 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb3fc-d3c0-4a49-9f3b-4ee856e84bee None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:15.882936 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb3fc-d3c0-4a49-9f3b-4ee856e84bee None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:15.883289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb3fc-d3c0-4a49-9f3b-4ee856e84bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:15.903241 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-238cb3fc-d3c0-4a49-9f3b-4ee856e84bee None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:15.904638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-238cb3fc-d3c0-4a49-9f3b-4ee856e84bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:15.918286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc14733-0e1a-433b-8edc-fcd3ba81d143 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.009604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d09fb-c4b8-4687-a0eb-8ca448872e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.010286 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d09fb-c4b8-4687-a0eb-8ca448872e8f admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.010570 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d09fb-c4b8-4687-a0eb-8ca448872e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.011042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d09fb-c4b8-4687-a0eb-8ca448872e8f admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.011789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d09fb-c4b8-4687-a0eb-8ca448872e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.014233 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d09fb-c4b8-4687-a0eb-8ca448872e8f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.042552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1377/2731] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:15 2022] POST /identity/v3/users => generated 295 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:16.048273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d09fb-c4b8-4687-a0eb-8ca448872e8f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.051184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1355/2732] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:16.055856 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-126d09fb-c4b8-4687-a0eb-8ca448872e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:16.058751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f9e39-b5ed-4c21-9a02-bc0aadabfea1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.059054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f9e39-b5ed-4c21-9a02-bc0aadabfea1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.059336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f9e39-b5ed-4c21-9a02-bc0aadabfea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.074170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d09fb-c4b8-4687-a0eb-8ca448872e8f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.080762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683319f0-a6b0-4ef9-82c0-91522c44598e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.082150 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683319f0-a6b0-4ef9-82c0-91522c44598e admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.082439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683319f0-a6b0-4ef9-82c0-91522c44598e admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.082718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683319f0-a6b0-4ef9-82c0-91522c44598e admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.083232 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-4e9f9e39-b5ed-4c21-9a02-bc0aadabfea1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:16.083552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683319f0-a6b0-4ef9-82c0-91522c44598e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.084782 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4e9f9e39-b5ed-4c21-9a02-bc0aadabfea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:16.085750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683319f0-a6b0-4ef9-82c0-91522c44598e admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.095280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1356/2733] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:16.099278 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1378/2734] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:16.100184 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683319f0-a6b0-4ef9-82c0-91522c44598e admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.101599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f9e39-b5ed-4c21-9a02-bc0aadabfea1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.111322 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f9e39-b5ed-4c21-9a02-bc0aadabfea1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.112465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683319f0-a6b0-4ef9-82c0-91522c44598e admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.116617 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b3d91-1a6f-4079-abd6-6fd24b803039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.117607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b3d91-1a6f-4079-abd6-6fd24b803039 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.117986 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b3d91-1a6f-4079-abd6-6fd24b803039 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.118364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b3d91-1a6f-4079-abd6-6fd24b803039 admin admin] PATH_INFO: `/v3/projects/ca7baa1f01b0446da3fabc79137c27c3/users/787fff587a4f4d58896c405cfaefefd0/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.120210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbbd22-11bd-44f1-b38a-7e54898cbe1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.121321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbbd22-11bd-44f1-b38a-7e54898cbe1f admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.121776 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbbd22-11bd-44f1-b38a-7e54898cbe1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.122468 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbbd22-11bd-44f1-b38a-7e54898cbe1f admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.123494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbbd22-11bd-44f1-b38a-7e54898cbe1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:02:16.124097 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbbd22-11bd-44f1-b38a-7e54898cbe1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.126122 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbbd22-11bd-44f1-b38a-7e54898cbe1f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.134723 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b3d91-1a6f-4079-abd6-6fd24b803039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7baa1f01b0446da3fabc79137c27c3, user_id=787fff587a4f4d58896c405cfaefefd0, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.137571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b3d91-1a6f-4079-abd6-6fd24b803039 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.141825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1379/2735] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 16:02:16.146594 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbbd22-11bd-44f1-b38a-7e54898cbe1f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.158104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbbd22-11bd-44f1-b38a-7e54898cbe1f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.160825 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1357/2736] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:16 2022] PUT /identity/v3/projects/ca7baa1f01b0446da3fabc79137c27c3/users/787fff587a4f4d58896c405cfaefefd0/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:16.165598 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b3d91-1a6f-4079-abd6-6fd24b803039 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.205367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:16.219836 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffdf02-2baa-45ed-8d38-5730313e24db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.220498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdf02-2baa-45ed-8d38-5730313e24db admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.220781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdf02-2baa-45ed-8d38-5730313e24db admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.221060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdf02-2baa-45ed-8d38-5730313e24db admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.221749 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffdf02-2baa-45ed-8d38-5730313e24db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.223711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffdf02-2baa-45ed-8d38-5730313e24db admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.229363 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b3d91-1a6f-4079-abd6-6fd24b803039 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.233982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd694588-975e-4a15-889c-996c83137e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.234647 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd694588-975e-4a15-889c-996c83137e6d admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.234957 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd694588-975e-4a15-889c-996c83137e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.235318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd694588-975e-4a15-889c-996c83137e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.235956 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd694588-975e-4a15-889c-996c83137e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.237533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd694588-975e-4a15-889c-996c83137e6d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.247904 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1358/2737] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:16.249988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1380/2738] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:16 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:02:16.252249 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd694588-975e-4a15-889c-996c83137e6d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.253745 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffdf02-2baa-45ed-8d38-5730313e24db admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.262961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd694588-975e-4a15-889c-996c83137e6d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.299966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:16.329550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffdf02-2baa-45ed-8d38-5730313e24db admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.331953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e8565-0123-4e9d-b92c-2d3f8cf4643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.332854 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e8565-0123-4e9d-b92c-2d3f8cf4643e admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.333145 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e8565-0123-4e9d-b92c-2d3f8cf4643e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.333512 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e8565-0123-4e9d-b92c-2d3f8cf4643e admin admin] PATH_INFO: `/v3/projects/ca7baa1f01b0446da3fabc79137c27c3/users/787fff587a4f4d58896c405cfaefefd0/roles/433478cc318a467c931f13a757f161fa` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.334848 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436fb53-06f2-4599-924e-8c273ecdd864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.336500 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e8565-0123-4e9d-b92c-2d3f8cf4643e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7baa1f01b0446da3fabc79137c27c3, user_id=787fff587a4f4d58896c405cfaefefd0, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.336660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436fb53-06f2-4599-924e-8c273ecdd864 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.336660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436fb53-06f2-4599-924e-8c273ecdd864 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.336950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436fb53-06f2-4599-924e-8c273ecdd864 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.337874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436fb53-06f2-4599-924e-8c273ecdd864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.339127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e8565-0123-4e9d-b92c-2d3f8cf4643e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.339792 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436fb53-06f2-4599-924e-8c273ecdd864 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.350449 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1359/2739] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:16 2022] PUT /identity/v3/projects/ca7baa1f01b0446da3fabc79137c27c3/users/787fff587a4f4d58896c405cfaefefd0/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:16.360360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1381/2740] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:16 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:02:16.360557 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1416791-35a4-4300-b8e3-62539e94695f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.360768 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1416791-35a4-4300-b8e3-62539e94695f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.361119 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1416791-35a4-4300-b8e3-62539e94695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.363953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436fb53-06f2-4599-924e-8c273ecdd864 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.373508 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436fb53-06f2-4599-924e-8c273ecdd864 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.389125 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-a1416791-35a4-4300-b8e3-62539e94695f None None] MFA Rules not processed for user `787fff587a4f4d58896c405cfaefefd0`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:16.390351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a1416791-35a4-4300-b8e3-62539e94695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:16.450860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5549d-e9b6-4169-94a1-18758ed6f9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.451750 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5549d-e9b6-4169-94a1-18758ed6f9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.451970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5549d-e9b6-4169-94a1-18758ed6f9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.452162 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5549d-e9b6-4169-94a1-18758ed6f9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.452888 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5549d-e9b6-4169-94a1-18758ed6f9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.454480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5549d-e9b6-4169-94a1-18758ed6f9d4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.465277 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1382/2741] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:16.468897 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5549d-e9b6-4169-94a1-18758ed6f9d4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.471800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1360/2742] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:16.481960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5549d-e9b6-4169-94a1-18758ed6f9d4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.484010 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1416791-35a4-4300-b8e3-62539e94695f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.485308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a1416791-35a4-4300-b8e3-62539e94695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:16.486820 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c7ef4-c9df-4993-ab12-b0902f352b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.487478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c7ef4-c9df-4993-ab12-b0902f352b39 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.487764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c7ef4-c9df-4993-ab12-b0902f352b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.488046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c7ef4-c9df-4993-ab12-b0902f352b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.488764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c7ef4-c9df-4993-ab12-b0902f352b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.490488 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c7ef4-c9df-4993-ab12-b0902f352b39 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.500681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1383/2743] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:16.504577 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c7ef4-c9df-4993-ab12-b0902f352b39 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.514784 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c7ef4-c9df-4993-ab12-b0902f352b39 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.519519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176bb16-13cc-4978-aede-448362657b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.520492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176bb16-13cc-4978-aede-448362657b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.520780 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176bb16-13cc-4978-aede-448362657b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.521053 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176bb16-13cc-4978-aede-448362657b6b admin admin] PATH_INFO: `/v3/projects/8b65a6fb6a6948b6854755c38791dbd3/users/9ee963a9f1664f1cb635180b288937c5/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.536654 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176bb16-13cc-4978-aede-448362657b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b65a6fb6a6948b6854755c38791dbd3, user_id=9ee963a9f1664f1cb635180b288937c5, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.539665 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176bb16-13cc-4978-aede-448362657b6b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.543343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1416791-35a4-4300-b8e3-62539e94695f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.547747 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e19b5-75b1-4bbd-98eb-fad80cfec134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.548396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e19b5-75b1-4bbd-98eb-fad80cfec134 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.548750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e19b5-75b1-4bbd-98eb-fad80cfec134 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.548966 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e19b5-75b1-4bbd-98eb-fad80cfec134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.551095 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7a3e19b5-75b1-4bbd-98eb-fad80cfec134 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:16.554633 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1384/2744] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:16 2022] PUT /identity/v3/projects/8b65a6fb6a6948b6854755c38791dbd3/users/9ee963a9f1664f1cb635180b288937c5/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:16.564688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c089d1a-c15d-403f-b465-18dd6ddb107b None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.564977 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c089d1a-c15d-403f-b465-18dd6ddb107b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.565255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c089d1a-c15d-403f-b465-18dd6ddb107b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.592485 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-6c089d1a-c15d-403f-b465-18dd6ddb107b None None] MFA Rules not processed for user `9ee963a9f1664f1cb635180b288937c5`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:16.593693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6c089d1a-c15d-403f-b465-18dd6ddb107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:16.605801 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:16.614134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e19b5-75b1-4bbd-98eb-fad80cfec134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.615615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e19b5-75b1-4bbd-98eb-fad80cfec134 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.625930 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1361/2745] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:02:16.662561 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1385/2746] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:02:16.667335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e19b5-75b1-4bbd-98eb-fad80cfec134 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.678273 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e19b5-75b1-4bbd-98eb-fad80cfec134 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.686928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:16.739315 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49839e-6283-469f-aedc-9da8fc981fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.739973 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49839e-6283-469f-aedc-9da8fc981fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.740254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49839e-6283-469f-aedc-9da8fc981fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.740529 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49839e-6283-469f-aedc-9da8fc981fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.741890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49839e-6283-469f-aedc-9da8fc981fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.743810 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49839e-6283-469f-aedc-9da8fc981fb5 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.769390 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1362/2747] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:16 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:16.773885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c089d1a-c15d-403f-b465-18dd6ddb107b None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.786363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c089d1a-c15d-403f-b465-18dd6ddb107b None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.882960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c31e0-679b-4a79-bb02-579a01c4cf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.885132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c31e0-679b-4a79-bb02-579a01c4cf0c admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.885871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c31e0-679b-4a79-bb02-579a01c4cf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.886768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c31e0-679b-4a79-bb02-579a01c4cf0c admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.888738 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c31e0-679b-4a79-bb02-579a01c4cf0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.892886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c31e0-679b-4a79-bb02-579a01c4cf0c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.938859 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1386/2748] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:16 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:16.939763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49839e-6283-469f-aedc-9da8fc981fb5 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.951136 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49839e-6283-469f-aedc-9da8fc981fb5 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.955869 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a90fd-f3af-4f0b-8ba4-ec8020b6a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.956538 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a90fd-f3af-4f0b-8ba4-ec8020b6a043 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.956824 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a90fd-f3af-4f0b-8ba4-ec8020b6a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.957106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a90fd-f3af-4f0b-8ba4-ec8020b6a043 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:16.957863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a90fd-f3af-4f0b-8ba4-ec8020b6a043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:16.959498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a90fd-f3af-4f0b-8ba4-ec8020b6a043 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:16.970814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1363/2749] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:02:16.974762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c31e0-679b-4a79-bb02-579a01c4cf0c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:16.988015 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c31e0-679b-4a79-bb02-579a01c4cf0c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:16.996399 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdd51c-b8fd-4dbd-8821-b53f988ae8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:16.997970 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdd51c-b8fd-4dbd-8821-b53f988ae8ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:16.998493 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdd51c-b8fd-4dbd-8821-b53f988ae8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:16.999048 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdd51c-b8fd-4dbd-8821-b53f988ae8ed admin admin] PATH_INFO: `/v3/projects/fdc33dd6a1814d2fab57ca6950b02905/users/b4e48c2077c948a59b4da72caa744ef4/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:17.016539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdd51c-b8fd-4dbd-8821-b53f988ae8ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc33dd6a1814d2fab57ca6950b02905, user_id=b4e48c2077c948a59b4da72caa744ef4, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:17.019456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdd51c-b8fd-4dbd-8821-b53f988ae8ed admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:17.032270 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1387/2750] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:16 2022] PUT /identity/v3/projects/fdc33dd6a1814d2fab57ca6950b02905/users/b4e48c2077c948a59b4da72caa744ef4/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:17.036251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a90fd-f3af-4f0b-8ba4-ec8020b6a043 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:17.046357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a90fd-f3af-4f0b-8ba4-ec8020b6a043 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:17.108650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c11b8-5b0b-452b-b754-1ecfde9caf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:17.109946 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c11b8-5b0b-452b-b754-1ecfde9caf56 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:17.110451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c11b8-5b0b-452b-b754-1ecfde9caf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:17.110979 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c11b8-5b0b-452b-b754-1ecfde9caf56 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:17.112289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c11b8-5b0b-452b-b754-1ecfde9caf56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:17.114066 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c11b8-5b0b-452b-b754-1ecfde9caf56 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:17.123814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1364/2751] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:17.128013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdd51c-b8fd-4dbd-8821-b53f988ae8ed admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:17.138657 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdd51c-b8fd-4dbd-8821-b53f988ae8ed admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:17.143252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea4b27-02a4-42d2-98c7-6d90221f3a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:17.144118 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea4b27-02a4-42d2-98c7-6d90221f3a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:17.144395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea4b27-02a4-42d2-98c7-6d90221f3a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:17.144663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea4b27-02a4-42d2-98c7-6d90221f3a36 admin admin] PATH_INFO: `/v3/projects/fdc33dd6a1814d2fab57ca6950b02905/users/b4e48c2077c948a59b4da72caa744ef4/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:17.146625 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea4b27-02a4-42d2-98c7-6d90221f3a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc33dd6a1814d2fab57ca6950b02905, user_id=b4e48c2077c948a59b4da72caa744ef4, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:17.149912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea4b27-02a4-42d2-98c7-6d90221f3a36 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:17.161348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1388/2752] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:17 2022] PUT /identity/v3/projects/fdc33dd6a1814d2fab57ca6950b02905/users/b4e48c2077c948a59b4da72caa744ef4/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:17.172607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd5c02-9f5f-47c0-b9f9-eafce7621fd1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:17.172907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd5c02-9f5f-47c0-b9f9-eafce7621fd1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:17.173221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd5c02-9f5f-47c0-b9f9-eafce7621fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:17.206777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-1acd5c02-9f5f-47c0-b9f9-eafce7621fd1 None None] MFA Rules not processed for user `b4e48c2077c948a59b4da72caa744ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:17.206777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-1acd5c02-9f5f-47c0-b9f9-eafce7621fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:17.277677 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1365/2753] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:17.290770 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea4b27-02a4-42d2-98c7-6d90221f3a36 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:17.349924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea4b27-02a4-42d2-98c7-6d90221f3a36 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:17.353765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a6b2f-cfb8-49ae-afcb-9afcadefd189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:17.354413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a6b2f-cfb8-49ae-afcb-9afcadefd189 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:17.354700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a6b2f-cfb8-49ae-afcb-9afcadefd189 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:17.355014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a6b2f-cfb8-49ae-afcb-9afcadefd189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:17.364545 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a6b2f-cfb8-49ae-afcb-9afcadefd189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:17.365992 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a6b2f-cfb8-49ae-afcb-9afcadefd189 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:17.385355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1389/2754] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:17.454060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd5c02-9f5f-47c0-b9f9-eafce7621fd1 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:17.463921 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd5c02-9f5f-47c0-b9f9-eafce7621fd1 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:17.467547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc8f13-9d76-4ae2-90a5-bca3f5e0bc18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:17.468211 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc8f13-9d76-4ae2-90a5-bca3f5e0bc18 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:17.468493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc8f13-9d76-4ae2-90a5-bca3f5e0bc18 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:17.468769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc8f13-9d76-4ae2-90a5-bca3f5e0bc18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:17.472846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:17.527246 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc8f13-9d76-4ae2-90a5-bca3f5e0bc18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:17.528599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc8f13-9d76-4ae2-90a5-bca3f5e0bc18 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:17.538005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1366/2755] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:17.797912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a6b2f-cfb8-49ae-afcb-9afcadefd189 service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:17.808244 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:17.860786 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a6b2f-cfb8-49ae-afcb-9afcadefd189 service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:17.865317 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:17.866176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:17.867093 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:17.867373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] PATH_INFO: `/v3/users/9ee963a9f1664f1cb635180b288937c5` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:17.868542 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee963a9f1664f1cb635180b288937c5)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:17.870452 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:17.913670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee963a9f1664f1cb635180b288937c5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:17.920714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee963a9f1664f1cb635180b288937c5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:17.927446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee963a9f1664f1cb635180b288937c5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:17.928083 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee963a9f1664f1cb635180b288937c5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:17.942196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee963a9f1664f1cb635180b288937c5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:17.948768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1390/2756] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:17 2022] DELETE /identity/v3/users/9ee963a9f1664f1cb635180b288937c5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:17.953507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21593435-b89e-4b97-b446-c5e637ed71c0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:17.953974 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21593435-b89e-4b97-b446-c5e637ed71c0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:17.954076 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21593435-b89e-4b97-b446-c5e637ed71c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:17.972233 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-21593435-b89e-4b97-b446-c5e637ed71c0 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:17.973455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-21593435-b89e-4b97-b446-c5e637ed71c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:18.046951 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1367/2757] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:18.057319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:18.058720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:18.115249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94b64f-3ce2-437e-8b12-954912dbedd4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:18.119554 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cddd2-a734-40d0-aab2-e27cd23d9e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:18.120193 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cddd2-a734-40d0-aab2-e27cd23d9e8c service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:18.120480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cddd2-a734-40d0-aab2-e27cd23d9e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:18.120763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cddd2-a734-40d0-aab2-e27cd23d9e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:18.130289 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cddd2-a734-40d0-aab2-e27cd23d9e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:18.131754 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cddd2-a734-40d0-aab2-e27cd23d9e8c service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:18.142113 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1391/2758] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:18.416025 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21593435-b89e-4b97-b446-c5e637ed71c0 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:18.418427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-21593435-b89e-4b97-b446-c5e637ed71c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:18.442675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21593435-b89e-4b97-b446-c5e637ed71c0 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:18.449168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:18.450464 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:18.450969 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:18.451595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] PATH_INFO: `/v3/projects/8b65a6fb6a6948b6854755c38791dbd3` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:18.453028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b65a6fb6a6948b6854755c38791dbd3)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:18.455112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:18.505291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b65a6fb6a6948b6854755c38791dbd3', 'request_id': 'req-6eac5681-4b2e-4e85-aa9a-9c331e187c97'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.512434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b65a6fb6a6948b6854755c38791dbd3', 'request_id': 'req-6eac5681-4b2e-4e85-aa9a-9c331e187c97'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.520599 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b65a6fb6a6948b6854755c38791dbd3', 'request_id': 'req-6eac5681-4b2e-4e85-aa9a-9c331e187c97'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.533099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] The token cache is being invalidate because project 8b65a6fb6a6948b6854755c38791dbd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:18.533545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.534218 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.535851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1368/2759] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:18 2022] DELETE /identity/v3/projects/8b65a6fb6a6948b6854755c38791dbd3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:18.540424 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cddd2-a734-40d0-aab2-e27cd23d9e8c service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:18.550655 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cddd2-a734-40d0-aab2-e27cd23d9e8c service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:18.583079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:18.613480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:18.614270 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:18.614550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:18.614865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] PATH_INFO: `/v3/users/b4e48c2077c948a59b4da72caa744ef4` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:18.616090 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e48c2077c948a59b4da72caa744ef4)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:18.618098 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:18.669003 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e48c2077c948a59b4da72caa744ef4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.677945 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e48c2077c948a59b4da72caa744ef4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.686066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e48c2077c948a59b4da72caa744ef4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.686838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e48c2077c948a59b4da72caa744ef4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.700581 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-70a8a121-1870-41bf-bd80-8cf78ef6e64a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e48c2077c948a59b4da72caa744ef4'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:18.706791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1392/2760] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:18 2022] DELETE /identity/v3/users/b4e48c2077c948a59b4da72caa744ef4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:02:18.981078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:18.982507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:19.003000 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac5681-4b2e-4e85-aa9a-9c331e187c97 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:19.034530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:19.068037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:19.068907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.069193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.070042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] PATH_INFO: `/v3/projects/fdc33dd6a1814d2fab57ca6950b02905` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.071254 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc33dd6a1814d2fab57ca6950b02905)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:19.073157 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:19.120901 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc33dd6a1814d2fab57ca6950b02905', 'request_id': 'req-97ca846c-2195-4160-a60a-77a65a86c231'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:19.127431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc33dd6a1814d2fab57ca6950b02905', 'request_id': 'req-97ca846c-2195-4160-a60a-77a65a86c231'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:19.137001 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc33dd6a1814d2fab57ca6950b02905', 'request_id': 'req-97ca846c-2195-4160-a60a-77a65a86c231'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:19.149724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] The token cache is being invalidate because project fdc33dd6a1814d2fab57ca6950b02905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:19.150050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:19.150707 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-97ca846c-2195-4160-a60a-77a65a86c231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:19.152427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1369/2761] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:18 2022] DELETE /identity/v3/projects/fdc33dd6a1814d2fab57ca6950b02905 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:19.158055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f7a8d-b46b-4d72-8c44-935ff48d7c73 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.158987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f7a8d-b46b-4d72-8c44-935ff48d7c73 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.159270 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f7a8d-b46b-4d72-8c44-935ff48d7c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.176922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-602f7a8d-b46b-4d72-8c44-935ff48d7c73 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:19.178226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-602f7a8d-b46b-4d72-8c44-935ff48d7c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:19.260353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1393/2762] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:19.268111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad95a1f-c691-4c4b-ba1a-0442bf8e3a2f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.268404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad95a1f-c691-4c4b-ba1a-0442bf8e3a2f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.268686 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad95a1f-c691-4c4b-ba1a-0442bf8e3a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.286866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-cad95a1f-c691-4c4b-ba1a-0442bf8e3a2f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:19.288827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cad95a1f-c691-4c4b-ba1a-0442bf8e3a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:19.296656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1370/2763] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:19.300861 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f7a8d-b46b-4d72-8c44-935ff48d7c73 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:19.311257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f7a8d-b46b-4d72-8c44-935ff48d7c73 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:19.315768 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a81610-38f4-4880-90fe-b17b0ec83357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:19.316415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81610-38f4-4880-90fe-b17b0ec83357 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.316696 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81610-38f4-4880-90fe-b17b0ec83357 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.316976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81610-38f4-4880-90fe-b17b0ec83357 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.317691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a81610-38f4-4880-90fe-b17b0ec83357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:02:19.318000 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a81610-38f4-4880-90fe-b17b0ec83357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:19.319661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a81610-38f4-4880-90fe-b17b0ec83357 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:19.331008 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1394/2764] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:02:19.335587 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad95a1f-c691-4c4b-ba1a-0442bf8e3a2f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:19.344744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad95a1f-c691-4c4b-ba1a-0442bf8e3a2f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:19.349414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0c14f-e47d-4f6a-8595-e053e4408c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:19.350059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0c14f-e47d-4f6a-8595-e053e4408c30 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.350343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0c14f-e47d-4f6a-8595-e053e4408c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.351174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0c14f-e47d-4f6a-8595-e053e4408c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.351902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0c14f-e47d-4f6a-8595-e053e4408c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:19.353832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0c14f-e47d-4f6a-8595-e053e4408c30 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:19.379792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1371/2765] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:19 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:19.383950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a81610-38f4-4880-90fe-b17b0ec83357 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:19.393938 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a81610-38f4-4880-90fe-b17b0ec83357 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:19.426607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:19.451598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185267b2-f7b9-4193-8ff6-86fceb03a5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:19.452204 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185267b2-f7b9-4193-8ff6-86fceb03a5cb admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.452494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185267b2-f7b9-4193-8ff6-86fceb03a5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.452775 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185267b2-f7b9-4193-8ff6-86fceb03a5cb admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.453477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185267b2-f7b9-4193-8ff6-86fceb03a5cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:19.455454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185267b2-f7b9-4193-8ff6-86fceb03a5cb admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:19.476168 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1395/2766] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:19 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:19.480517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0c14f-e47d-4f6a-8595-e053e4408c30 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:19.490955 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0c14f-e47d-4f6a-8595-e053e4408c30 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:19.496207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083b75b-6b98-49f5-a740-f8aa9f93302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:19.496871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083b75b-6b98-49f5-a740-f8aa9f93302d admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.497160 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083b75b-6b98-49f5-a740-f8aa9f93302d admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.497445 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083b75b-6b98-49f5-a740-f8aa9f93302d admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.498174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083b75b-6b98-49f5-a740-f8aa9f93302d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:19.499808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083b75b-6b98-49f5-a740-f8aa9f93302d admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:19.510213 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1372/2767] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:19.513976 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185267b2-f7b9-4193-8ff6-86fceb03a5cb admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:19.523962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185267b2-f7b9-4193-8ff6-86fceb03a5cb admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:19.528787 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c93cbe-7a54-4bf5-953e-54cf90193b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:19.529439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c93cbe-7a54-4bf5-953e-54cf90193b56 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.529727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c93cbe-7a54-4bf5-953e-54cf90193b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.530006 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c93cbe-7a54-4bf5-953e-54cf90193b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.530785 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c93cbe-7a54-4bf5-953e-54cf90193b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:19.532406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c93cbe-7a54-4bf5-953e-54cf90193b56 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:19.543101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1396/2768] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:19.546908 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083b75b-6b98-49f5-a740-f8aa9f93302d admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:19.557751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083b75b-6b98-49f5-a740-f8aa9f93302d admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:19.562349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49145c02-3a19-47b0-8a63-78f904d32e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:19.563258 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49145c02-3a19-47b0-8a63-78f904d32e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.563547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49145c02-3a19-47b0-8a63-78f904d32e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.563830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49145c02-3a19-47b0-8a63-78f904d32e4e admin admin] PATH_INFO: `/v3/projects/e61f6e5e764e40cd877bed2f3a328fab/users/aec23bdfa3ce4aa0af2970a0a6fd4e9d/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.577928 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49145c02-3a19-47b0-8a63-78f904d32e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e61f6e5e764e40cd877bed2f3a328fab, user_id=aec23bdfa3ce4aa0af2970a0a6fd4e9d, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:19.580596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49145c02-3a19-47b0-8a63-78f904d32e4e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:19.593143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1373/2769] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:19 2022] PUT /identity/v3/projects/e61f6e5e764e40cd877bed2f3a328fab/users/aec23bdfa3ce4aa0af2970a0a6fd4e9d/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:19.598023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd48b2-2ea1-495e-8535-32947350b8ea None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.598309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd48b2-2ea1-495e-8535-32947350b8ea None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.598584 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd48b2-2ea1-495e-8535-32947350b8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.616831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-05fd48b2-2ea1-495e-8535-32947350b8ea None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:19.618021 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-05fd48b2-2ea1-495e-8535-32947350b8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:19.674789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1397/2770] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:19.688215 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49145c02-3a19-47b0-8a63-78f904d32e4e admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:19.690944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-49145c02-3a19-47b0-8a63-78f904d32e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:19.748787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49145c02-3a19-47b0-8a63-78f904d32e4e admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:19.752414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ca676-4894-40d9-9a78-a5eb831d959c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:19.753061 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ca676-4894-40d9-9a78-a5eb831d959c service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:19.753345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ca676-4894-40d9-9a78-a5eb831d959c service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:19.753624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ca676-4894-40d9-9a78-a5eb831d959c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:19.762458 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ca676-4894-40d9-9a78-a5eb831d959c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:19.763916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ca676-4894-40d9-9a78-a5eb831d959c service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:19.767763 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:19.775298 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1374/2771] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:20.424549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e659c7d-80f9-4823-b606-b208c45f51df None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:20.425046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e659c7d-80f9-4823-b606-b208c45f51df None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:20.425268 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e659c7d-80f9-4823-b606-b208c45f51df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:20.456405 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-3e659c7d-80f9-4823-b606-b208c45f51df None None] MFA Rules not processed for user `aec23bdfa3ce4aa0af2970a0a6fd4e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:20.457685 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3e659c7d-80f9-4823-b606-b208c45f51df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:20.525628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1398/2772] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:20.530124 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ca676-4894-40d9-9a78-a5eb831d959c service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:20.540545 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ca676-4894-40d9-9a78-a5eb831d959c service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:20.545255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29d660-a2ca-4f0f-b414-111350956df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:20.545907 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29d660-a2ca-4f0f-b414-111350956df9 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:20.546195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29d660-a2ca-4f0f-b414-111350956df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:20.546475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29d660-a2ca-4f0f-b414-111350956df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:20.547217 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29d660-a2ca-4f0f-b414-111350956df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:20.549137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29d660-a2ca-4f0f-b414-111350956df9 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:20.579187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1375/2773] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:02:20 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:20.580652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e659c7d-80f9-4823-b606-b208c45f51df None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:20.591863 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e659c7d-80f9-4823-b606-b208c45f51df None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:20.636022 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:20.651054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398d4b3-bf63-4c93-b890-38517470b1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:20.651727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398d4b3-bf63-4c93-b890-38517470b1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:20.652014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398d4b3-bf63-4c93-b890-38517470b1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:20.652295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398d4b3-bf63-4c93-b890-38517470b1c7 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:20.652982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398d4b3-bf63-4c93-b890-38517470b1c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:20.654932 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398d4b3-bf63-4c93-b890-38517470b1c7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:20.675217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1399/2774] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:02:20 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:02:20.679174 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29d660-a2ca-4f0f-b414-111350956df9 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:20.689396 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29d660-a2ca-4f0f-b414-111350956df9 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:20.693889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788d098-80e0-47bf-9aae-695c47a94475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:20.694537 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788d098-80e0-47bf-9aae-695c47a94475 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:20.694855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788d098-80e0-47bf-9aae-695c47a94475 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:20.695134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788d098-80e0-47bf-9aae-695c47a94475 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:20.695850 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788d098-80e0-47bf-9aae-695c47a94475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:20.697444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788d098-80e0-47bf-9aae-695c47a94475 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:20.708711 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1376/2775] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:20.713279 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398d4b3-bf63-4c93-b890-38517470b1c7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:20.723203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398d4b3-bf63-4c93-b890-38517470b1c7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:20.728454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d215e83-fa6d-4c2e-ac1a-caa9bdee4880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:20.729320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d215e83-fa6d-4c2e-ac1a-caa9bdee4880 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:20.729604 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d215e83-fa6d-4c2e-ac1a-caa9bdee4880 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:20.729883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d215e83-fa6d-4c2e-ac1a-caa9bdee4880 admin admin] PATH_INFO: `/v3/projects/932c7c8f9b12437298d97b5469ed094a/users/01f1b3f4417f445d8c75d058dfb97f5b/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:20.744377 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d215e83-fa6d-4c2e-ac1a-caa9bdee4880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=932c7c8f9b12437298d97b5469ed094a, user_id=01f1b3f4417f445d8c75d058dfb97f5b, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:20.747101 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d215e83-fa6d-4c2e-ac1a-caa9bdee4880 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:20.760473 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1400/2776] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:20 2022] PUT /identity/v3/projects/932c7c8f9b12437298d97b5469ed094a/users/01f1b3f4417f445d8c75d058dfb97f5b/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:20.764346 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788d098-80e0-47bf-9aae-695c47a94475 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:20.774579 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788d098-80e0-47bf-9aae-695c47a94475 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:20.829021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b9bd7-78e6-4528-b8d0-bd08d5396b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:20.829679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b9bd7-78e6-4528-b8d0-bd08d5396b8b admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:20.829963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b9bd7-78e6-4528-b8d0-bd08d5396b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:20.830239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b9bd7-78e6-4528-b8d0-bd08d5396b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:20.830977 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b9bd7-78e6-4528-b8d0-bd08d5396b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:20.832563 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b9bd7-78e6-4528-b8d0-bd08d5396b8b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:20.842646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1377/2777] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:02:20.846662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d215e83-fa6d-4c2e-ac1a-caa9bdee4880 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:20.856711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d215e83-fa6d-4c2e-ac1a-caa9bdee4880 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:20.861321 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc466a-09f3-48d9-9c35-46b8e3939087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:20.862182 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc466a-09f3-48d9-9c35-46b8e3939087 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:20.862462 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc466a-09f3-48d9-9c35-46b8e3939087 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:20.862752 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc466a-09f3-48d9-9c35-46b8e3939087 admin admin] PATH_INFO: `/v3/projects/932c7c8f9b12437298d97b5469ed094a/users/01f1b3f4417f445d8c75d058dfb97f5b/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:20.864718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc466a-09f3-48d9-9c35-46b8e3939087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=932c7c8f9b12437298d97b5469ed094a, user_id=01f1b3f4417f445d8c75d058dfb97f5b, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:20.867353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc466a-09f3-48d9-9c35-46b8e3939087 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:20.877167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1401/2778] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:02:20 2022] PUT /identity/v3/projects/932c7c8f9b12437298d97b5469ed094a/users/01f1b3f4417f445d8c75d058dfb97f5b/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:21.678101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488868f-076c-4630-b806-ebd77f6ef469 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:21.678615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488868f-076c-4630-b806-ebd77f6ef469 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:21.678773 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488868f-076c-4630-b806-ebd77f6ef469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:21.707734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-4488868f-076c-4630-b806-ebd77f6ef469 None None] MFA Rules not processed for user `01f1b3f4417f445d8c75d058dfb97f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:21.708958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4488868f-076c-4630-b806-ebd77f6ef469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:21.780886 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1378/2779] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:21 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:21.793448 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc466a-09f3-48d9-9c35-46b8e3939087 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:21.795304 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-1dfc466a-09f3-48d9-9c35-46b8e3939087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:21.853746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc466a-09f3-48d9-9c35-46b8e3939087 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:21.857461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78da91-1636-43ce-a9b2-a73353cb4e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:21.858108 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78da91-1636-43ce-a9b2-a73353cb4e0e service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:21.858393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78da91-1636-43ce-a9b2-a73353cb4e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:21.858669 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78da91-1636-43ce-a9b2-a73353cb4e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:21.915067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78da91-1636-43ce-a9b2-a73353cb4e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:21.916502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78da91-1636-43ce-a9b2-a73353cb4e0e service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:21.925212 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1402/2780] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:24.385247 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488868f-076c-4630-b806-ebd77f6ef469 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:24.386692 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-4488868f-076c-4630-b806-ebd77f6ef469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:24.419879 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488868f-076c-4630-b806-ebd77f6ef469 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:24.423439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcaa145-8a9e-473a-9d22-4a636db40a42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:24.424140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaa145-8a9e-473a-9d22-4a636db40a42 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:24.424482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaa145-8a9e-473a-9d22-4a636db40a42 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:24.424755 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaa145-8a9e-473a-9d22-4a636db40a42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:24.426486 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7fcaa145-8a9e-473a-9d22-4a636db40a42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:24.451369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:24.508615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcaa145-8a9e-473a-9d22-4a636db40a42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:24.510041 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcaa145-8a9e-473a-9d22-4a636db40a42 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:24.519343 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1379/2781] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:25.327218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78da91-1636-43ce-a9b2-a73353cb4e0e service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:25.328691 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bf78da91-1636-43ce-a9b2-a73353cb4e0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:25.362268 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcaa145-8a9e-473a-9d22-4a636db40a42 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:25.371616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcaa145-8a9e-473a-9d22-4a636db40a42 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:25.375317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f5d19-ae1b-4a43-80e6-0933185a423a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:25.375958 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f5d19-ae1b-4a43-80e6-0933185a423a service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:25.376243 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f5d19-ae1b-4a43-80e6-0933185a423a service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:25.376522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f5d19-ae1b-4a43-80e6-0933185a423a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:25.385610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f5d19-ae1b-4a43-80e6-0933185a423a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:25.387054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f5d19-ae1b-4a43-80e6-0933185a423a service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:25.389413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:25.397042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1380/2782] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:02:25.424664 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78da91-1636-43ce-a9b2-a73353cb4e0e service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:25.428406 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:25.429650 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:25.429927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:25.430200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:25.432038 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:25.515023 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:25.516442 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:25.527579 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1403/2783] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:35.392451 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=33 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:35.522499 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=40 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:56.121682 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f5d19-ae1b-4a43-80e6-0933185a423a service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:56.131224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f5d19-ae1b-4a43-80e6-0933185a423a service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:56.134885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329e7d5-5d9c-4479-b204-87720d448cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:56.135547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329e7d5-5d9c-4479-b204-87720d448cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:56.135830 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329e7d5-5d9c-4479-b204-87720d448cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:56.136107 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329e7d5-5d9c-4479-b204-87720d448cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:56.137841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c329e7d5-5d9c-4479-b204-87720d448cb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:56.202224 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329e7d5-5d9c-4479-b204-87720d448cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:56.203699 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329e7d5-5d9c-4479-b204-87720d448cb9 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:56.213666 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1381/2784] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:58.822550 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:58.824595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:58.948980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcde86f-f879-4c06-aca1-0d35ff05dc21 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:58.956441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:58.957732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:58.957855 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:58.958050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] PATH_INFO: `/v3/users/dedef0d8de0142149f72a46371099af5` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:58.959355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dedef0d8de0142149f72a46371099af5)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:58.962030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:59.024079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dedef0d8de0142149f72a46371099af5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.033481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dedef0d8de0142149f72a46371099af5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.046030 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dedef0d8de0142149f72a46371099af5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.047296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dedef0d8de0142149f72a46371099af5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.068831 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dedef0d8de0142149f72a46371099af5'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.075319 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1404/2785] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:58 2022] DELETE /identity/v3/users/dedef0d8de0142149f72a46371099af5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:59.080314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced05fb-f5b4-4650-bc32-cdae3cf14ad3 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:59.080765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced05fb-f5b4-4650-bc32-cdae3cf14ad3 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:59.080861 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced05fb-f5b4-4650-bc32-cdae3cf14ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:59.113022 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-0ced05fb-f5b4-4650-bc32-cdae3cf14ad3 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:02:59.114371 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0ced05fb-f5b4-4650-bc32-cdae3cf14ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:59.188882 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1382/2786] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:02:59.203578 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:59.205202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:59.290698 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295b62b-c692-45a3-a38e-1b2500150cfb admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:59.295318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f216c4-0270-4c8f-8be9-e77e4a235e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:59.296080 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f216c4-0270-4c8f-8be9-e77e4a235e50 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:59.296367 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f216c4-0270-4c8f-8be9-e77e4a235e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:59.296802 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f216c4-0270-4c8f-8be9-e77e4a235e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:59.310112 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f216c4-0270-4c8f-8be9-e77e4a235e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:59.312686 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f216c4-0270-4c8f-8be9-e77e4a235e50 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:59.325643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1405/2787] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:02:59.675062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced05fb-f5b4-4650-bc32-cdae3cf14ad3 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:59.676935 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0ced05fb-f5b4-4650-bc32-cdae3cf14ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:02:59.689827 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced05fb-f5b4-4650-bc32-cdae3cf14ad3 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:59.694601 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:59.695514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:59.695800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:59.696081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] PATH_INFO: `/v3/projects/fe7d80ef8e1643fe98e7b0d9c2222f26` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:59.697287 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe7d80ef8e1643fe98e7b0d9c2222f26)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:59.699482 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:59.758130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe7d80ef8e1643fe98e7b0d9c2222f26', 'request_id': 'req-d5104b3d-c714-429e-9d79-9cac221d0f62'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.765185 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe7d80ef8e1643fe98e7b0d9c2222f26', 'request_id': 'req-d5104b3d-c714-429e-9d79-9cac221d0f62'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.776048 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe7d80ef8e1643fe98e7b0d9c2222f26', 'request_id': 'req-d5104b3d-c714-429e-9d79-9cac221d0f62'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.793479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] The token cache is being invalidate because project fe7d80ef8e1643fe98e7b0d9c2222f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:02:59.793952 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.794628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.796819 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1383/2788] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:02:59 2022] DELETE /identity/v3/projects/fe7d80ef8e1643fe98e7b0d9c2222f26 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:02:59.800949 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f216c4-0270-4c8f-8be9-e77e4a235e50 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:02:59.808079 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:02:59.813267 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f216c4-0270-4c8f-8be9-e77e4a235e50 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:02:59.882078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:02:59.883736 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:02:59.884196 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:02:59.884624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] PATH_INFO: `/v3/users/787fff587a4f4d58896c405cfaefefd0` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:02:59.886026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] RBAC: Authorizing `identity:delete_user(user_id=787fff587a4f4d58896c405cfaefefd0)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:02:59.888498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:02:59.950524 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '787fff587a4f4d58896c405cfaefefd0'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.964457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '787fff587a4f4d58896c405cfaefefd0'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.977257 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '787fff587a4f4d58896c405cfaefefd0'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:02:59.978120 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '787fff587a4f4d58896c405cfaefefd0'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:00.000817 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-32355306-47c0-49ed-bb5c-79c041cb531a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '787fff587a4f4d58896c405cfaefefd0'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:00.011415 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1406/2789] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:02:59 2022] DELETE /identity/v3/users/787fff587a4f4d58896c405cfaefefd0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:00.516034 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:00.517498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:00.534368 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5104b3d-c714-429e-9d79-9cac221d0f62 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:00.544595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:00.633844 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:00.635110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:00.635554 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:00.636023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] PATH_INFO: `/v3/projects/ca7baa1f01b0446da3fabc79137c27c3` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:00.637480 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca7baa1f01b0446da3fabc79137c27c3)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:00.640251 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:00.715144 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca7baa1f01b0446da3fabc79137c27c3', 'request_id': 'req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:00.730615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca7baa1f01b0446da3fabc79137c27c3', 'request_id': 'req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:00.746021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca7baa1f01b0446da3fabc79137c27c3', 'request_id': 'req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:00.760751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] The token cache is being invalidate because project ca7baa1f01b0446da3fabc79137c27c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:03:00.761446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:00.762626 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-30e2ed40-4e25-46f4-9b23-8c1f38a95ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:00.768660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1384/2790] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:03:00 2022] DELETE /identity/v3/projects/ca7baa1f01b0446da3fabc79137c27c3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:00.770634 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4b763-60f6-4824-b55c-733716867ce7 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:00.771905 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4b763-60f6-4824-b55c-733716867ce7 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:00.772362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4b763-60f6-4824-b55c-733716867ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:00.827363 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-bfb4b763-60f6-4824-b55c-733716867ce7 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:00.828599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bfb4b763-60f6-4824-b55c-733716867ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:00.893167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:00.911533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1407/2791] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:00.913485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6001de1-78ab-4fac-b679-529dae2b6c78 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:00.913909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6001de1-78ab-4fac-b679-529dae2b6c78 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:00.914555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6001de1-78ab-4fac-b679-529dae2b6c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:00.934995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c6001de1-78ab-4fac-b679-529dae2b6c78 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:00.936777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c6001de1-78ab-4fac-b679-529dae2b6c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:00.945786 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1385/2792] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:00.950074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4b763-60f6-4824-b55c-733716867ce7 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:00.961115 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4b763-60f6-4824-b55c-733716867ce7 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:00.966409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e948d8-c715-44b8-a9d9-9c08a4c5164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:00.966701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e948d8-c715-44b8-a9d9-9c08a4c5164a admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:00.967104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e948d8-c715-44b8-a9d9-9c08a4c5164a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:00.967466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e948d8-c715-44b8-a9d9-9c08a4c5164a admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:00.968379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e948d8-c715-44b8-a9d9-9c08a4c5164a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:03:00.968756 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e948d8-c715-44b8-a9d9-9c08a4c5164a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:00.970430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e948d8-c715-44b8-a9d9-9c08a4c5164a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:00.989213 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1408/2793] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:03:00.989758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6001de1-78ab-4fac-b679-529dae2b6c78 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.000609 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6001de1-78ab-4fac-b679-529dae2b6c78 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.005468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc658b0-9d6c-46e8-949c-af6d804c675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.006141 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc658b0-9d6c-46e8-949c-af6d804c675f admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.006424 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc658b0-9d6c-46e8-949c-af6d804c675f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.006704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc658b0-9d6c-46e8-949c-af6d804c675f admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.007429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc658b0-9d6c-46e8-949c-af6d804c675f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.009631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc658b0-9d6c-46e8-949c-af6d804c675f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.041351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1386/2794] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:03:00 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:01.046203 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e948d8-c715-44b8-a9d9-9c08a4c5164a admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.058056 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e948d8-c715-44b8-a9d9-9c08a4c5164a admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.117050 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b9ba-5005-4780-a8b3-4ce141cca6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.117726 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b9ba-5005-4780-a8b3-4ce141cca6fb admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.118012 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b9ba-5005-4780-a8b3-4ce141cca6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.118293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b9ba-5005-4780-a8b3-4ce141cca6fb admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.119132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609b9ba-5005-4780-a8b3-4ce141cca6fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.121064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609b9ba-5005-4780-a8b3-4ce141cca6fb admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.141701 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1409/2795] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:03:01 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:01.146112 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc658b0-9d6c-46e8-949c-af6d804c675f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.156914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc658b0-9d6c-46e8-949c-af6d804c675f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.162183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e30eb7-0f57-4cc4-9ce9-3f15a78bea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.162896 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e30eb7-0f57-4cc4-9ce9-3f15a78bea82 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.163184 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e30eb7-0f57-4cc4-9ce9-3f15a78bea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.163469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e30eb7-0f57-4cc4-9ce9-3f15a78bea82 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.164567 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e30eb7-0f57-4cc4-9ce9-3f15a78bea82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.166130 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e30eb7-0f57-4cc4-9ce9-3f15a78bea82 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.177843 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1387/2796] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:01.181799 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b9ba-5005-4780-a8b3-4ce141cca6fb admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.192571 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b9ba-5005-4780-a8b3-4ce141cca6fb admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.198683 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94c4f8-86b1-4fa2-b497-6fd51541d257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.199457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94c4f8-86b1-4fa2-b497-6fd51541d257 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.199968 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94c4f8-86b1-4fa2-b497-6fd51541d257 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.200288 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94c4f8-86b1-4fa2-b497-6fd51541d257 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.201340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94c4f8-86b1-4fa2-b497-6fd51541d257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.203223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94c4f8-86b1-4fa2-b497-6fd51541d257 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.218643 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1410/2797] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:01.220374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e30eb7-0f57-4cc4-9ce9-3f15a78bea82 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.226652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94c4f8-86b1-4fa2-b497-6fd51541d257 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.228523 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cd94c4f8-86b1-4fa2-b497-6fd51541d257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:01.235596 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e30eb7-0f57-4cc4-9ce9-3f15a78bea82 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.240517 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0414-2a1a-402d-a7bf-c4bfd0193e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.241438 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0414-2a1a-402d-a7bf-c4bfd0193e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.241637 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94c4f8-86b1-4fa2-b497-6fd51541d257 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.241769 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0414-2a1a-402d-a7bf-c4bfd0193e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.241990 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0414-2a1a-402d-a7bf-c4bfd0193e94 admin admin] PATH_INFO: `/v3/projects/a7206e340c144239b1932899979599e2/users/66d83ac3915a4e6ba738f3720c087c2e/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.246243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.247130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.247410 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.247681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] PATH_INFO: `/v3/users/aec23bdfa3ce4aa0af2970a0a6fd4e9d` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.249722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec23bdfa3ce4aa0af2970a0a6fd4e9d)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.251653 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.257985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c0414-2a1a-402d-a7bf-c4bfd0193e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7206e340c144239b1932899979599e2, user_id=66d83ac3915a4e6ba738f3720c087c2e, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.261291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c0414-2a1a-402d-a7bf-c4bfd0193e94 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.283613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1388/2798] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:03:01 2022] PUT /identity/v3/projects/a7206e340c144239b1932899979599e2/users/66d83ac3915a4e6ba738f3720c087c2e/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:01.295823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b36e5-4f2e-456b-9d9d-607265bec59d None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.296135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b36e5-4f2e-456b-9d9d-607265bec59d None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.296404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b36e5-4f2e-456b-9d9d-607265bec59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.301207 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec23bdfa3ce4aa0af2970a0a6fd4e9d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.310032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec23bdfa3ce4aa0af2970a0a6fd4e9d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.317742 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec23bdfa3ce4aa0af2970a0a6fd4e9d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.318392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec23bdfa3ce4aa0af2970a0a6fd4e9d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.324014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f56b36e5-4f2e-456b-9d9d-607265bec59d None None] MFA Rules not processed for user `66d83ac3915a4e6ba738f3720c087c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:01.325361 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f56b36e5-4f2e-456b-9d9d-607265bec59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:01.330765 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec23bdfa3ce4aa0af2970a0a6fd4e9d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.336869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1411/2799] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:03:01 2022] DELETE /identity/v3/users/aec23bdfa3ce4aa0af2970a0a6fd4e9d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:01.352277 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:01.406491 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1389/2800] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:01.446472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.448209 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:01.493033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:01.510045 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5810a8b-4757-4342-bacb-9901d83042e4 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.514688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f408323-a88f-4b6b-a3bb-f6aac681c89c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.515360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f408323-a88f-4b6b-a3bb-f6aac681c89c service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.515641 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f408323-a88f-4b6b-a3bb-f6aac681c89c service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.515918 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f408323-a88f-4b6b-a3bb-f6aac681c89c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.522523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b36e5-4f2e-456b-9d9d-607265bec59d None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.525713 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f56b36e5-4f2e-456b-9d9d-607265bec59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:01.588275 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f408323-a88f-4b6b-a3bb-f6aac681c89c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.589714 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f408323-a88f-4b6b-a3bb-f6aac681c89c service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.599627 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1412/2801] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:03:01.606165 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b36e5-4f2e-456b-9d9d-607265bec59d None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.614663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f408323-a88f-4b6b-a3bb-f6aac681c89c service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.616057 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3f408323-a88f-4b6b-a3bb-f6aac681c89c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:01.668660 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.669533 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.669822 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.670099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] PATH_INFO: `/v3/projects/e61f6e5e764e40cd877bed2f3a328fab` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.671360 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e61f6e5e764e40cd877bed2f3a328fab)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.673377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.696215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f408323-a88f-4b6b-a3bb-f6aac681c89c service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.702220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.703503 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.703860 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.704170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] PATH_INFO: `/v3/users/66d83ac3915a4e6ba738f3720c087c2e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.705531 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d83ac3915a4e6ba738f3720c087c2e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.707611 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.722912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e61f6e5e764e40cd877bed2f3a328fab', 'request_id': 'req-48526345-36e5-4463-a4b1-c29f0e24d0cc'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.729421 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e61f6e5e764e40cd877bed2f3a328fab', 'request_id': 'req-48526345-36e5-4463-a4b1-c29f0e24d0cc'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.740702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e61f6e5e764e40cd877bed2f3a328fab', 'request_id': 'req-48526345-36e5-4463-a4b1-c29f0e24d0cc'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.753133 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] The token cache is being invalidate because project e61f6e5e764e40cd877bed2f3a328fab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:03:01.753444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.754072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.755668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1390/2802] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:03:01 2022] DELETE /identity/v3/projects/e61f6e5e764e40cd877bed2f3a328fab => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:01.755958 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d83ac3915a4e6ba738f3720c087c2e'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.762595 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.764681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:01.769875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d83ac3915a4e6ba738f3720c087c2e'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.771662 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:01.778310 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d83ac3915a4e6ba738f3720c087c2e'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.779032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d83ac3915a4e6ba738f3720c087c2e'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.792070 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-1e0712eb-63c2-4292-bb5b-72ca23fa8d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d83ac3915a4e6ba738f3720c087c2e'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.798190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1413/2803] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:03:01 2022] DELETE /identity/v3/users/66d83ac3915a4e6ba738f3720c087c2e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:03:01.805034 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67a1a5-f0b1-48a0-9b03-34e1fd610ff9 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.805325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67a1a5-f0b1-48a0-9b03-34e1fd610ff9 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.805612 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67a1a5-f0b1-48a0-9b03-34e1fd610ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.823443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9d67a1a5-f0b1-48a0-9b03-34e1fd610ff9 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:01.824662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9d67a1a5-f0b1-48a0-9b03-34e1fd610ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:01.830663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:01.843469 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48526345-36e5-4463-a4b1-c29f0e24d0cc admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.887451 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1414/2804] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:01.898098 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67a1a5-f0b1-48a0-9b03-34e1fd610ff9 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:01.900172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9d67a1a5-f0b1-48a0-9b03-34e1fd610ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:01.911603 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.912494 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.912801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.913078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] PATH_INFO: `/v3/users/01f1b3f4417f445d8c75d058dfb97f5b` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.914916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f1b3f4417f445d8c75d058dfb97f5b)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.916980 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.966546 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f1b3f4417f445d8c75d058dfb97f5b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.967980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67a1a5-f0b1-48a0-9b03-34e1fd610ff9 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:01.971766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fcb40-8147-4e66-b0f7-ed12acb8ddac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:01.972481 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fcb40-8147-4e66-b0f7-ed12acb8ddac service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:01.972813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fcb40-8147-4e66-b0f7-ed12acb8ddac service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:01.973139 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fcb40-8147-4e66-b0f7-ed12acb8ddac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:01.974757 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f1b3f4417f445d8c75d058dfb97f5b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.982069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f1b3f4417f445d8c75d058dfb97f5b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.983336 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f1b3f4417f445d8c75d058dfb97f5b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.983708 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fcb40-8147-4e66-b0f7-ed12acb8ddac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:01.984103 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f1b3f4417f445d8c75d058dfb97f5b'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:01.985284 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fcb40-8147-4e66-b0f7-ed12acb8ddac service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:01.995443 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1415/2805] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:03:02.003036 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1391/2806] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:03:01 2022] DELETE /identity/v3/users/01f1b3f4417f445d8c75d058dfb97f5b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:03:02.288629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fcb40-8147-4e66-b0f7-ed12acb8ddac service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.290375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-153fcb40-8147-4e66-b0f7-ed12acb8ddac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:02.303986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fcb40-8147-4e66-b0f7-ed12acb8ddac service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.362862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.364600 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:02.387906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.389062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.389280 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.389559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] PATH_INFO: `/v3/projects/932c7c8f9b12437298d97b5469ed094a` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.391315 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=932c7c8f9b12437298d97b5469ed094a)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.393217 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.412905 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:02.446142 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932c7c8f9b12437298d97b5469ed094a', 'request_id': 'req-6ccd12ed-7701-42a0-b6c2-734077717590'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.449991 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c7a9a-ce11-499a-a811-af4c52f7b741 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.452630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932c7c8f9b12437298d97b5469ed094a', 'request_id': 'req-6ccd12ed-7701-42a0-b6c2-734077717590'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.455297 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:02.461089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932c7c8f9b12437298d97b5469ed094a', 'request_id': 'req-6ccd12ed-7701-42a0-b6c2-734077717590'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.473089 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] The token cache is being invalidate because project 932c7c8f9b12437298d97b5469ed094a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:03:02.473379 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.474024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6ccd12ed-7701-42a0-b6c2-734077717590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.475601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1416/2807] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:03:02 2022] DELETE /identity/v3/projects/932c7c8f9b12437298d97b5469ed094a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:02.480225 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a002c34-3e68-43e8-b006-c126176d9c02 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.480511 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a002c34-3e68-43e8-b006-c126176d9c02 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.480793 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a002c34-3e68-43e8-b006-c126176d9c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.499072 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5a002c34-3e68-43e8-b006-c126176d9c02 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:02.500294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5a002c34-3e68-43e8-b006-c126176d9c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:02.511700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.512556 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.512837 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.513111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] PATH_INFO: `/v3/projects/a7206e340c144239b1932899979599e2` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.514515 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7206e340c144239b1932899979599e2)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.516547 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.562373 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1417/2808] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:02.563705 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7206e340c144239b1932899979599e2', 'request_id': 'req-f620bbff-2e4a-411d-8be8-9489d9fac99e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.569809 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7206e340c144239b1932899979599e2', 'request_id': 'req-f620bbff-2e4a-411d-8be8-9489d9fac99e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.570540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6ca58-9f13-4e25-b893-911e461b49d4 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.570862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6ca58-9f13-4e25-b893-911e461b49d4 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.571143 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6ca58-9f13-4e25-b893-911e461b49d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.577414 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7206e340c144239b1932899979599e2', 'request_id': 'req-f620bbff-2e4a-411d-8be8-9489d9fac99e'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.588325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] The token cache is being invalidate because project a7206e340c144239b1932899979599e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:03:02.588775 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.589410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-f620bbff-2e4a-411d-8be8-9489d9fac99e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:02.589567 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-8da6ca58-9f13-4e25-b893-911e461b49d4 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:02.590630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-8da6ca58-9f13-4e25-b893-911e461b49d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:02.591181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1392/2809] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:03:02 2022] DELETE /identity/v3/projects/a7206e340c144239b1932899979599e2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:03:02.596395 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20042d76-d782-4055-9567-720d52e8f1e1 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.596734 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20042d76-d782-4055-9567-720d52e8f1e1 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.597060 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20042d76-d782-4055-9567-720d52e8f1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.614997 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-20042d76-d782-4055-9567-720d52e8f1e1 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:02.616192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-20042d76-d782-4055-9567-720d52e8f1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:02.650767 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1418/2810] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:02.655420 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6ca58-9f13-4e25-b893-911e461b49d4 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.664521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6ca58-9f13-4e25-b893-911e461b49d4 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.669400 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7c191-7ebb-42bb-b54c-73e3da9ed33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.670123 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7c191-7ebb-42bb-b54c-73e3da9ed33c admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.670403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7c191-7ebb-42bb-b54c-73e3da9ed33c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.670688 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7c191-7ebb-42bb-b54c-73e3da9ed33c admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.671435 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7c191-7ebb-42bb-b54c-73e3da9ed33c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:03:02.671740 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7c191-7ebb-42bb-b54c-73e3da9ed33c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.673347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7c191-7ebb-42bb-b54c-73e3da9ed33c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.676114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1393/2811] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:02.683746 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab0e4d-9730-45b9-82a5-51f4a5bacb9b None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.684039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab0e4d-9730-45b9-82a5-51f4a5bacb9b None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.684317 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab0e4d-9730-45b9-82a5-51f4a5bacb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.688841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1419/2812] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 16:03:02.689733 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7c191-7ebb-42bb-b54c-73e3da9ed33c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.699640 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7c191-7ebb-42bb-b54c-73e3da9ed33c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.702573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-8aab0e4d-9730-45b9-82a5-51f4a5bacb9b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:02.703801 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-8aab0e4d-9730-45b9-82a5-51f4a5bacb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:02.706014 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af8be0-6012-41a1-8360-c5b0bc4ff183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.707121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af8be0-6012-41a1-8360-c5b0bc4ff183 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.707679 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af8be0-6012-41a1-8360-c5b0bc4ff183 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.708145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af8be0-6012-41a1-8360-c5b0bc4ff183 admin admin] PATH_INFO: `/v3/projects` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.709016 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af8be0-6012-41a1-8360-c5b0bc4ff183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.711076 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af8be0-6012-41a1-8360-c5b0bc4ff183 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.711739 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1394/2813] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:02.715780 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab0e4d-9730-45b9-82a5-51f4a5bacb9b None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.720839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:02.727212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab0e4d-9730-45b9-82a5-51f4a5bacb9b None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.734252 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8bc9c-9a87-4047-91a8-8629b1b4c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.734989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8bc9c-9a87-4047-91a8-8629b1b4c008 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.735281 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8bc9c-9a87-4047-91a8-8629b1b4c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.735607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8bc9c-9a87-4047-91a8-8629b1b4c008 admin admin] PATH_INFO: `/v3/domains` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.736410 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8bc9c-9a87-4047-91a8-8629b1b4c008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:03:02.736728 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8bc9c-9a87-4047-91a8-8629b1b4c008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.738365 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8bc9c-9a87-4047-91a8-8629b1b4c008 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.743236 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1420/2814] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:03:02 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:03:02.747190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af8be0-6012-41a1-8360-c5b0bc4ff183 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.756906 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1395/2815] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 16:03:02.757840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8bc9c-9a87-4047-91a8-8629b1b4c008 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.767945 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8bc9c-9a87-4047-91a8-8629b1b4c008 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.799062 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:02.811592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af8be0-6012-41a1-8360-c5b0bc4ff183 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.817899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15e60a-f9e8-48d6-a3e5-dc3f5adc8081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.818598 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e60a-f9e8-48d6-a3e5-dc3f5adc8081 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.818917 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e60a-f9e8-48d6-a3e5-dc3f5adc8081 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.819199 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e60a-f9e8-48d6-a3e5-dc3f5adc8081 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.819885 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15e60a-f9e8-48d6-a3e5-dc3f5adc8081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.821866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15e60a-f9e8-48d6-a3e5-dc3f5adc8081 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.826338 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb944bb-778b-4188-bac7-5e72417595ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.827027 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb944bb-778b-4188-bac7-5e72417595ec admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.827318 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb944bb-778b-4188-bac7-5e72417595ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.827597 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb944bb-778b-4188-bac7-5e72417595ec admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.828444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb944bb-778b-4188-bac7-5e72417595ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.830357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb944bb-778b-4188-bac7-5e72417595ec admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.841206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1421/2816] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:03:02 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:03:02.845176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15e60a-f9e8-48d6-a3e5-dc3f5adc8081 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.855190 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15e60a-f9e8-48d6-a3e5-dc3f5adc8081 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.860335 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1396/2817] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:03:02 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:02.861255 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb944bb-778b-4188-bac7-5e72417595ec admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.911471 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66841113-089c-4e74-8ac6-5be21dc5ff04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.912063 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66841113-089c-4e74-8ac6-5be21dc5ff04 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.912347 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66841113-089c-4e74-8ac6-5be21dc5ff04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.912624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66841113-089c-4e74-8ac6-5be21dc5ff04 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.913348 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66841113-089c-4e74-8ac6-5be21dc5ff04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.915247 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66841113-089c-4e74-8ac6-5be21dc5ff04 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.921558 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb944bb-778b-4188-bac7-5e72417595ec admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.926353 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1422/2818] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:03:02.926646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84014722-8d83-45a3-ba9f-298c75ec1817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.927350 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84014722-8d83-45a3-ba9f-298c75ec1817 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.927640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84014722-8d83-45a3-ba9f-298c75ec1817 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.927916 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84014722-8d83-45a3-ba9f-298c75ec1817 admin admin] PATH_INFO: `/v3/users` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.928590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84014722-8d83-45a3-ba9f-298c75ec1817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.930313 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66841113-089c-4e74-8ac6-5be21dc5ff04 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.930476 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84014722-8d83-45a3-ba9f-298c75ec1817 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.942716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66841113-089c-4e74-8ac6-5be21dc5ff04 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.947960 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9f5d-387f-474c-8bd6-542b9c8d5119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.948673 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9f5d-387f-474c-8bd6-542b9c8d5119 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.948959 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9f5d-387f-474c-8bd6-542b9c8d5119 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.949235 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9f5d-387f-474c-8bd6-542b9c8d5119 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.949961 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9f5d-387f-474c-8bd6-542b9c8d5119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.951487 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1397/2819] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:03:02 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:03:02.954914 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9f5d-387f-474c-8bd6-542b9c8d5119 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.955293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84014722-8d83-45a3-ba9f-298c75ec1817 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.966316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1423/2820] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:02.969148 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84014722-8d83-45a3-ba9f-298c75ec1817 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.970441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9f5d-387f-474c-8bd6-542b9c8d5119 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:02.974029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8111f-5e6c-4f29-8a4d-751c4094ccc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.974714 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8111f-5e6c-4f29-8a4d-751c4094ccc6 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.975030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8111f-5e6c-4f29-8a4d-751c4094ccc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.975306 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8111f-5e6c-4f29-8a4d-751c4094ccc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.976024 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8111f-5e6c-4f29-8a4d-751c4094ccc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:02.977634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8111f-5e6c-4f29-8a4d-751c4094ccc6 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:02.980252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9f5d-387f-474c-8bd6-542b9c8d5119 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:02.985641 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f6719-b943-4fdb-8ee7-a72b9165ce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:02.986496 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f6719-b943-4fdb-8ee7-a72b9165ce2f admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:02.986808 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f6719-b943-4fdb-8ee7-a72b9165ce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:02.987085 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f6719-b943-4fdb-8ee7-a72b9165ce2f admin admin] PATH_INFO: `/v3/projects/45b42b4a43894f5fa1502766dfedec94/users/f41c2a67fd1d47c89c8b8875e41970c9/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:02.993722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1398/2821] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:03:02.994777 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8111f-5e6c-4f29-8a4d-751c4094ccc6 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:03.002522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f6719-b943-4fdb-8ee7-a72b9165ce2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=45b42b4a43894f5fa1502766dfedec94, user_id=f41c2a67fd1d47c89c8b8875e41970c9, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:03.004508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8111f-5e6c-4f29-8a4d-751c4094ccc6 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:03.005585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f6719-b943-4fdb-8ee7-a72b9165ce2f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:03.009192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65472c5-e3bc-4976-9202-63b9756d9923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:03.009764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65472c5-e3bc-4976-9202-63b9756d9923 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:03.010045 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65472c5-e3bc-4976-9202-63b9756d9923 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:03.010319 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65472c5-e3bc-4976-9202-63b9756d9923 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:03.011074 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65472c5-e3bc-4976-9202-63b9756d9923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:03.012846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65472c5-e3bc-4976-9202-63b9756d9923 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:03.021423 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1424/2822] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:03:02 2022] PUT /identity/v3/projects/45b42b4a43894f5fa1502766dfedec94/users/f41c2a67fd1d47c89c8b8875e41970c9/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:03:03.024357 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1399/2823] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 16:03:03.028559 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f6719-b943-4fdb-8ee7-a72b9165ce2f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:03.032377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aa9f2-7c20-4335-a752-0a2b9ff2d90f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:03.032685 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aa9f2-7c20-4335-a752-0a2b9ff2d90f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:03.032982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aa9f2-7c20-4335-a752-0a2b9ff2d90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:03.059086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-ba5aa9f2-7c20-4335-a752-0a2b9ff2d90f None None] MFA Rules not processed for user `f41c2a67fd1d47c89c8b8875e41970c9`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:03.060294 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ba5aa9f2-7c20-4335-a752-0a2b9ff2d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:03.089121 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:03.090320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f6719-b943-4fdb-8ee7-a72b9165ce2f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:03.096700 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471befd0-9260-4d6b-a24e-7aab8686e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:03.097502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471befd0-9260-4d6b-a24e-7aab8686e1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:03.097782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471befd0-9260-4d6b-a24e-7aab8686e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:03.098058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471befd0-9260-4d6b-a24e-7aab8686e1a7 admin admin] PATH_INFO: `/v3/projects/3149517a9f0d4f528aac091dce32b615/users/4b8d636a0ff145188a69b1393906cf89/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:03.113166 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471befd0-9260-4d6b-a24e-7aab8686e1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3149517a9f0d4f528aac091dce32b615, user_id=4b8d636a0ff145188a69b1393906cf89, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:03.115873 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471befd0-9260-4d6b-a24e-7aab8686e1a7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:03.127307 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1400/2824] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:03.129036 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1425/2825] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:03:03 2022] PUT /identity/v3/projects/3149517a9f0d4f528aac091dce32b615/users/4b8d636a0ff145188a69b1393906cf89/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:03:03.139406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5aa9f2-7c20-4335-a752-0a2b9ff2d90f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:03.140159 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce93b4f-ec18-439c-bd25-0f79dc99a740 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:03.140449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce93b4f-ec18-439c-bd25-0f79dc99a740 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:03.140727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce93b4f-ec18-439c-bd25-0f79dc99a740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:03.140832 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ba5aa9f2-7c20-4335-a752-0a2b9ff2d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:03.167300 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-fce93b4f-ec18-439c-bd25-0f79dc99a740 None None] MFA Rules not processed for user `4b8d636a0ff145188a69b1393906cf89`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:03.168413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fce93b4f-ec18-439c-bd25-0f79dc99a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:03.180505 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:03.199511 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5aa9f2-7c20-4335-a752-0a2b9ff2d90f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:03.233522 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1426/2826] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:03.244605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce93b4f-ec18-439c-bd25-0f79dc99a740 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:03.245969 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-fce93b4f-ec18-439c-bd25-0f79dc99a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:03.250521 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260746a-d171-4a04-b207-6ce71bd0470e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:03.251206 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260746a-d171-4a04-b207-6ce71bd0470e service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:03.251497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260746a-d171-4a04-b207-6ce71bd0470e service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:03.251779 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260746a-d171-4a04-b207-6ce71bd0470e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:03.257596 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce93b4f-ec18-439c-bd25-0f79dc99a740 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:03.261327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3240ccf-d9fb-4cb2-bcf8-892218f95f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:03.261951 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3240ccf-d9fb-4cb2-bcf8-892218f95f6e service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:03.262227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3240ccf-d9fb-4cb2-bcf8-892218f95f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:03.262498 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3240ccf-d9fb-4cb2-bcf8-892218f95f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:03.270805 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3240ccf-d9fb-4cb2-bcf8-892218f95f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:03.272227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3240ccf-d9fb-4cb2-bcf8-892218f95f6e service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:03.281519 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1427/2827] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:03:03.318002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260746a-d171-4a04-b207-6ce71bd0470e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:03.319503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260746a-d171-4a04-b207-6ce71bd0470e service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:03.329553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1401/2828] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:03:03.349341 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3240ccf-d9fb-4cb2-bcf8-892218f95f6e service cinder] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:03.416094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3240ccf-d9fb-4cb2-bcf8-892218f95f6e service cinder] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:03.425066 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:03.426560 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:03.427130 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:03.427670 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] PATH_INFO: `/v3/users/f41c2a67fd1d47c89c8b8875e41970c9` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:03.429540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f41c2a67fd1d47c89c8b8875e41970c9)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:03.431599 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:03.480174 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f41c2a67fd1d47c89c8b8875e41970c9'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:03.487715 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f41c2a67fd1d47c89c8b8875e41970c9'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:03.496397 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f41c2a67fd1d47c89c8b8875e41970c9'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:03.497133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f41c2a67fd1d47c89c8b8875e41970c9'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:03.511513 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f41c2a67fd1d47c89c8b8875e41970c9'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:03.518430 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1428/2829] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:03:03 2022] DELETE /identity/v3/users/f41c2a67fd1d47c89c8b8875e41970c9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:03.523049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b415a0-8a1c-4c3b-aded-6c5feb9e25ed None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:03.523442 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b415a0-8a1c-4c3b-aded-6c5feb9e25ed None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:03.523931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b415a0-8a1c-4c3b-aded-6c5feb9e25ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:03.543455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-90b415a0-8a1c-4c3b-aded-6c5feb9e25ed None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:03.544735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-90b415a0-8a1c-4c3b-aded-6c5feb9e25ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:03.591091 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=100 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:03.614668 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1402/2830] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:03.625256 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:03.627100 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:03.649876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:03.687386 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b05832-c971-4e43-887c-0a385db3fc82 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:03.690919 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4634109-1dc3-49d5-b4ff-ac9756953d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:03.691552 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634109-1dc3-49d5-b4ff-ac9756953d11 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:03.691832 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634109-1dc3-49d5-b4ff-ac9756953d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:03.692129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634109-1dc3-49d5-b4ff-ac9756953d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:03.701838 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4634109-1dc3-49d5-b4ff-ac9756953d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:03.703403 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4634109-1dc3-49d5-b4ff-ac9756953d11 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:03.714480 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1429/2831] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:03:03.987871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b415a0-8a1c-4c3b-aded-6c5feb9e25ed None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:03.989301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-90b415a0-8a1c-4c3b-aded-6c5feb9e25ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:04.001995 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b415a0-8a1c-4c3b-aded-6c5feb9e25ed None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:04.006749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:04.007633 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.007915 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.008193 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] PATH_INFO: `/v3/projects/45b42b4a43894f5fa1502766dfedec94` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.010030 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45b42b4a43894f5fa1502766dfedec94)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:04.012072 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:04.060280 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45b42b4a43894f5fa1502766dfedec94', 'request_id': 'req-172673ff-a18c-4a1c-99f5-12717fa086d7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:04.066963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45b42b4a43894f5fa1502766dfedec94', 'request_id': 'req-172673ff-a18c-4a1c-99f5-12717fa086d7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:04.075064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45b42b4a43894f5fa1502766dfedec94', 'request_id': 'req-172673ff-a18c-4a1c-99f5-12717fa086d7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:04.085873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] The token cache is being invalidate because project 45b42b4a43894f5fa1502766dfedec94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:03:04.086159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:04.086826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-172673ff-a18c-4a1c-99f5-12717fa086d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:04.088372 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1403/2832] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:03:03 2022] DELETE /identity/v3/projects/45b42b4a43894f5fa1502766dfedec94 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:04.093242 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888ac9d-45be-42b6-ae5a-dfd47f184e42 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.093716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888ac9d-45be-42b6-ae5a-dfd47f184e42 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.093825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888ac9d-45be-42b6-ae5a-dfd47f184e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.111654 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-f888ac9d-45be-42b6-ae5a-dfd47f184e42 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:04.112853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f888ac9d-45be-42b6-ae5a-dfd47f184e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:04.172398 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1430/2833] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:04.180019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e19d85-8ff4-4d2e-b954-f3615a4d6ceb None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.180310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e19d85-8ff4-4d2e-b954-f3615a4d6ceb None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.180592 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e19d85-8ff4-4d2e-b954-f3615a4d6ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.198888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-f4e19d85-8ff4-4d2e-b954-f3615a4d6ceb None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:04.200002 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-f4e19d85-8ff4-4d2e-b954-f3615a4d6ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:04.207816 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1404/2834] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:04.211791 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888ac9d-45be-42b6-ae5a-dfd47f184e42 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:04.221322 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888ac9d-45be-42b6-ae5a-dfd47f184e42 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:04.225807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0799ad-8bd2-43e3-bab7-84efb5e105d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:04.226445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0799ad-8bd2-43e3-bab7-84efb5e105d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.226723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0799ad-8bd2-43e3-bab7-84efb5e105d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.227028 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0799ad-8bd2-43e3-bab7-84efb5e105d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.227737 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0799ad-8bd2-43e3-bab7-84efb5e105d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:03:04.228040 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0799ad-8bd2-43e3-bab7-84efb5e105d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:04.229628 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0799ad-8bd2-43e3-bab7-84efb5e105d7 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:04.241327 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1431/2835] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 16:03:04.245302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e19d85-8ff4-4d2e-b954-f3615a4d6ceb None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:04.255139 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e19d85-8ff4-4d2e-b954-f3615a4d6ceb None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:04.260077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81064ab-d158-4d5b-ac4c-fb4208b63e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:04.260342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81064ab-d158-4d5b-ac4c-fb4208b63e0a admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.260628 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81064ab-d158-4d5b-ac4c-fb4208b63e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.261043 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81064ab-d158-4d5b-ac4c-fb4208b63e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.261726 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81064ab-d158-4d5b-ac4c-fb4208b63e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:04.263635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81064ab-d158-4d5b-ac4c-fb4208b63e0a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:04.288302 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1405/2836] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:03:04 2022] POST /identity/v3/projects => generated 399 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:04.292671 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0799ad-8bd2-43e3-bab7-84efb5e105d7 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:04.303392 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0799ad-8bd2-43e3-bab7-84efb5e105d7 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:04.363224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186d5f6-83ef-440e-8790-1d502abe75f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:04.363895 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d5f6-83ef-440e-8790-1d502abe75f3 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.364180 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d5f6-83ef-440e-8790-1d502abe75f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.364458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d5f6-83ef-440e-8790-1d502abe75f3 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.365187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186d5f6-83ef-440e-8790-1d502abe75f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:04.367145 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186d5f6-83ef-440e-8790-1d502abe75f3 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:04.387682 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1432/2837] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:03:04 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:04.391758 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81064ab-d158-4d5b-ac4c-fb4208b63e0a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:04.401992 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81064ab-d158-4d5b-ac4c-fb4208b63e0a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:04.406634 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5616d-298b-46ff-8afa-ece02486dcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:04.407320 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5616d-298b-46ff-8afa-ece02486dcb2 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.407607 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5616d-298b-46ff-8afa-ece02486dcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.407888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5616d-298b-46ff-8afa-ece02486dcb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.408613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5616d-298b-46ff-8afa-ece02486dcb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:04.410188 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5616d-298b-46ff-8afa-ece02486dcb2 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:04.422067 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1406/2838] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:04.425111 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186d5f6-83ef-440e-8790-1d502abe75f3 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:04.436675 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186d5f6-83ef-440e-8790-1d502abe75f3 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:04.441502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e377-fad0-43af-9a79-7aebf05ec900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:04.442202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e377-fad0-43af-9a79-7aebf05ec900 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.442491 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e377-fad0-43af-9a79-7aebf05ec900 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.442924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e377-fad0-43af-9a79-7aebf05ec900 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.443661 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e377-fad0-43af-9a79-7aebf05ec900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:04.445335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e377-fad0-43af-9a79-7aebf05ec900 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:04.458221 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1433/2839] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:04.462472 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5616d-298b-46ff-8afa-ece02486dcb2 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:04.473253 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5616d-298b-46ff-8afa-ece02486dcb2 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:04.478099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd374d51-7da7-4657-a6d9-9751b09fcaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:04.479017 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd374d51-7da7-4657-a6d9-9751b09fcaa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.479314 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd374d51-7da7-4657-a6d9-9751b09fcaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.479584 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd374d51-7da7-4657-a6d9-9751b09fcaa7 admin admin] PATH_INFO: `/v3/projects/f2dfdeaca66f4ed3b50bc38fea4ba77d/users/2ab5f9a1f4a7412dae7f28ed0799d2dc/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.494418 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd374d51-7da7-4657-a6d9-9751b09fcaa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2dfdeaca66f4ed3b50bc38fea4ba77d, user_id=2ab5f9a1f4a7412dae7f28ed0799d2dc, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:04.497086 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd374d51-7da7-4657-a6d9-9751b09fcaa7 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:04.510064 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1407/2840] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:03:04 2022] PUT /identity/v3/projects/f2dfdeaca66f4ed3b50bc38fea4ba77d/users/2ab5f9a1f4a7412dae7f28ed0799d2dc/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:04.521697 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e8280-cef6-4e34-82b2-2c18be485d48 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.521986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e8280-cef6-4e34-82b2-2c18be485d48 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.522293 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e8280-cef6-4e34-82b2-2c18be485d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.550761 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-9e6e8280-cef6-4e34-82b2-2c18be485d48 None None] MFA Rules not processed for user `2ab5f9a1f4a7412dae7f28ed0799d2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:04.551972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9e6e8280-cef6-4e34-82b2-2c18be485d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:04.568474 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:04.626606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1434/2841] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:04.634005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd374d51-7da7-4657-a6d9-9751b09fcaa7 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:04.635865 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-fd374d51-7da7-4657-a6d9-9751b09fcaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:04.695386 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd374d51-7da7-4657-a6d9-9751b09fcaa7 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:04.700265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb9c7b-b00a-4fc8-be4d-392ef5cd2fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:04.701362 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb9c7b-b00a-4fc8-be4d-392ef5cd2fbb service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:04.701573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb9c7b-b00a-4fc8-be4d-392ef5cd2fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:04.701789 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb9c7b-b00a-4fc8-be4d-392ef5cd2fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:04.711470 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb9c7b-b00a-4fc8-be4d-392ef5cd2fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:04.712985 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb9c7b-b00a-4fc8-be4d-392ef5cd2fbb service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:04.724054 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1408/2842] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:03:14.618458 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=19 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:14.718702 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:22.087272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e8280-cef6-4e34-82b2-2c18be485d48 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:22.148763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e8280-cef6-4e34-82b2-2c18be485d48 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:22.153273 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:22.154132 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:22.154417 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:22.154693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] PATH_INFO: `/v3/users/2ab5f9a1f4a7412dae7f28ed0799d2dc` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:22.156614 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab5f9a1f4a7412dae7f28ed0799d2dc)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:22.158534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:22.204454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab5f9a1f4a7412dae7f28ed0799d2dc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.211511 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab5f9a1f4a7412dae7f28ed0799d2dc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.218997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab5f9a1f4a7412dae7f28ed0799d2dc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.219624 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab5f9a1f4a7412dae7f28ed0799d2dc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.231325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab5f9a1f4a7412dae7f28ed0799d2dc'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.237153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1435/2843] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:03:22 2022] DELETE /identity/v3/users/2ab5f9a1f4a7412dae7f28ed0799d2dc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:22.241639 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d4c50-a3bb-488e-8c36-5fb383e3bd0f None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:22.242099 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d4c50-a3bb-488e-8c36-5fb383e3bd0f None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:22.242216 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d4c50-a3bb-488e-8c36-5fb383e3bd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:22.259493 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-7c6d4c50-a3bb-488e-8c36-5fb383e3bd0f None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:22.260750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7c6d4c50-a3bb-488e-8c36-5fb383e3bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:22.319530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1409/2844] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:22.330409 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:22.331871 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:22.389636 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ebfc3-254b-4e30-ace9-ebbf8ae16eaf admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:22.393303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92e21d-295e-452d-a550-9e718361436a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:22.393943 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92e21d-295e-452d-a550-9e718361436a service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:22.394227 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92e21d-295e-452d-a550-9e718361436a service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:22.394506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92e21d-295e-452d-a550-9e718361436a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:22.403742 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92e21d-295e-452d-a550-9e718361436a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:22.405125 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92e21d-295e-452d-a550-9e718361436a service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:22.414782 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1436/2845] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:03:22.678295 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d4c50-a3bb-488e-8c36-5fb383e3bd0f None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:22.679745 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7c6d4c50-a3bb-488e-8c36-5fb383e3bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:22.692168 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d4c50-a3bb-488e-8c36-5fb383e3bd0f None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:22.696871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:22.697814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:22.698104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:22.698509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] PATH_INFO: `/v3/projects/f2dfdeaca66f4ed3b50bc38fea4ba77d` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:22.699735 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2dfdeaca66f4ed3b50bc38fea4ba77d)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:22.701629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:22.748902 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2dfdeaca66f4ed3b50bc38fea4ba77d', 'request_id': 'req-c68808bd-9057-4cb4-a6c2-7cb2790b709a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.755098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2dfdeaca66f4ed3b50bc38fea4ba77d', 'request_id': 'req-c68808bd-9057-4cb4-a6c2-7cb2790b709a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.762571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2dfdeaca66f4ed3b50bc38fea4ba77d', 'request_id': 'req-c68808bd-9057-4cb4-a6c2-7cb2790b709a'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.772873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] The token cache is being invalidate because project f2dfdeaca66f4ed3b50bc38fea4ba77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:03:22.773192 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.773813 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-c68808bd-9057-4cb4-a6c2-7cb2790b709a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:22.775383 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1410/2846] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:03:22 2022] DELETE /identity/v3/projects/f2dfdeaca66f4ed3b50bc38fea4ba77d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:22.780797 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bcd6f-cfcf-4e1f-ad4b-08caec7567f8 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:22.781081 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bcd6f-cfcf-4e1f-ad4b-08caec7567f8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:22.781382 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bcd6f-cfcf-4e1f-ad4b-08caec7567f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:22.800054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-a50bcd6f-cfcf-4e1f-ad4b-08caec7567f8 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:22.801246 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a50bcd6f-cfcf-4e1f-ad4b-08caec7567f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:22.817899 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:22.862979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1437/2847] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:22.870847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bf365-67ec-4357-80f6-c168ab528ea9 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:22.871221 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bf365-67ec-4357-80f6-c168ab528ea9 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:22.871573 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bf365-67ec-4357-80f6-c168ab528ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:22.890891 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-142bf365-67ec-4357-80f6-c168ab528ea9 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:22.892078 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-142bf365-67ec-4357-80f6-c168ab528ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:22.899833 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1411/2848] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:22.903773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bcd6f-cfcf-4e1f-ad4b-08caec7567f8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:22.914055 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bcd6f-cfcf-4e1f-ad4b-08caec7567f8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:22.918825 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db2a8c-eb46-4cd8-9880-04ccbf38410c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:22.919469 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db2a8c-eb46-4cd8-9880-04ccbf38410c admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:22.919746 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db2a8c-eb46-4cd8-9880-04ccbf38410c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:22.920020 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db2a8c-eb46-4cd8-9880-04ccbf38410c admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:22.920809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db2a8c-eb46-4cd8-9880-04ccbf38410c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:03:22.921060 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db2a8c-eb46-4cd8-9880-04ccbf38410c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:22.922642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db2a8c-eb46-4cd8-9880-04ccbf38410c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:22.934226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1438/2849] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:03:22.937988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bf365-67ec-4357-80f6-c168ab528ea9 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:22.947478 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bf365-67ec-4357-80f6-c168ab528ea9 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:22.952731 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1135b2-aa14-4765-9cc4-cb05e7311338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:22.953382 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1135b2-aa14-4765-9cc4-cb05e7311338 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:22.953681 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1135b2-aa14-4765-9cc4-cb05e7311338 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:22.953970 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1135b2-aa14-4765-9cc4-cb05e7311338 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:22.954670 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1135b2-aa14-4765-9cc4-cb05e7311338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:22.956624 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1135b2-aa14-4765-9cc4-cb05e7311338 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:22.982575 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1412/2850] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:03:22 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:22.986723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db2a8c-eb46-4cd8-9880-04ccbf38410c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:22.996982 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db2a8c-eb46-4cd8-9880-04ccbf38410c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:23.054502 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d20457-cec4-41a7-8759-e85d82aebe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:23.055434 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d20457-cec4-41a7-8759-e85d82aebe8b admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:23.055649 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d20457-cec4-41a7-8759-e85d82aebe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:23.055934 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d20457-cec4-41a7-8759-e85d82aebe8b admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:23.056621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d20457-cec4-41a7-8759-e85d82aebe8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:23.058555 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d20457-cec4-41a7-8759-e85d82aebe8b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:23.082152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1439/2851] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:03:22 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:23.086541 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1135b2-aa14-4765-9cc4-cb05e7311338 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:23.099201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1135b2-aa14-4765-9cc4-cb05e7311338 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:23.103929 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1771dc0-9156-4e25-aac0-f1e9f59aa11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:23.104585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1771dc0-9156-4e25-aac0-f1e9f59aa11f admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:23.104866 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1771dc0-9156-4e25-aac0-f1e9f59aa11f admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:23.105142 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1771dc0-9156-4e25-aac0-f1e9f59aa11f admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:23.105858 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1771dc0-9156-4e25-aac0-f1e9f59aa11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:23.107632 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1771dc0-9156-4e25-aac0-f1e9f59aa11f admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:23.118706 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1413/2852] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:23.122986 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d20457-cec4-41a7-8759-e85d82aebe8b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:23.142841 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d20457-cec4-41a7-8759-e85d82aebe8b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:23.148375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47954922-e18d-4a4f-9603-955baa298af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:23.149043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47954922-e18d-4a4f-9603-955baa298af2 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:23.149354 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47954922-e18d-4a4f-9603-955baa298af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:23.149630 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47954922-e18d-4a4f-9603-955baa298af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:23.150357 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47954922-e18d-4a4f-9603-955baa298af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:23.152002 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47954922-e18d-4a4f-9603-955baa298af2 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:23.170600 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1440/2853] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:23.171840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1771dc0-9156-4e25-aac0-f1e9f59aa11f admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:23.183256 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1771dc0-9156-4e25-aac0-f1e9f59aa11f admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:23.188379 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635310aa-2b0f-4003-9293-4da8c24d519a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:23.189264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635310aa-2b0f-4003-9293-4da8c24d519a admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:23.189548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635310aa-2b0f-4003-9293-4da8c24d519a admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:23.189823 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635310aa-2b0f-4003-9293-4da8c24d519a admin admin] PATH_INFO: `/v3/projects/202b681b581b4f6ea4fe9fcbf049c185/users/34e42d4cebe14642a71209ddd2b25fd2/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:23.204848 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635310aa-2b0f-4003-9293-4da8c24d519a admin admin] RBAC: Authorizing `identity:create_grant(project_id=202b681b581b4f6ea4fe9fcbf049c185, user_id=34e42d4cebe14642a71209ddd2b25fd2, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:23.207648 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635310aa-2b0f-4003-9293-4da8c24d519a admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:23.219988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1414/2854] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:03:23 2022] PUT /identity/v3/projects/202b681b581b4f6ea4fe9fcbf049c185/users/34e42d4cebe14642a71209ddd2b25fd2/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:03:23.225047 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5146124-fd91-433d-b6e4-a1027f18cb7a None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:23.225362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5146124-fd91-433d-b6e4-a1027f18cb7a None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:23.225753 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5146124-fd91-433d-b6e4-a1027f18cb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:23.243074 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-c5146124-fd91-433d-b6e4-a1027f18cb7a None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:23.244291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-c5146124-fd91-433d-b6e4-a1027f18cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:23.302226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1441/2855] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:23.312483 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635310aa-2b0f-4003-9293-4da8c24d519a admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:23.314276 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-635310aa-2b0f-4003-9293-4da8c24d519a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:23.336454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:23.373050 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635310aa-2b0f-4003-9293-4da8c24d519a admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:23.377154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b044c-c1b8-46ee-b50a-e459eaf330fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:23.377787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b044c-c1b8-46ee-b50a-e459eaf330fd service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:23.378068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b044c-c1b8-46ee-b50a-e459eaf330fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:23.378341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b044c-c1b8-46ee-b50a-e459eaf330fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:23.387039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b044c-c1b8-46ee-b50a-e459eaf330fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:23.388450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b044c-c1b8-46ee-b50a-e459eaf330fd service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:23.398345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1415/2856] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:03:27.219921 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff214e1f-6783-44ea-a0ff-bcd5eeb1c76f None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:27.220413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff214e1f-6783-44ea-a0ff-bcd5eeb1c76f None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:27.220506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff214e1f-6783-44ea-a0ff-bcd5eeb1c76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:27.247981 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ff214e1f-6783-44ea-a0ff-bcd5eeb1c76f None None] MFA Rules not processed for user `34e42d4cebe14642a71209ddd2b25fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:27.249294 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ff214e1f-6783-44ea-a0ff-bcd5eeb1c76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:27.272916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:27.327472 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1442/2857] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:27.332800 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b044c-c1b8-46ee-b50a-e459eaf330fd service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:27.344591 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b044c-c1b8-46ee-b50a-e459eaf330fd service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:27.355555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdfe0a-b35a-4101-bfab-a75a140a6baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:27.356237 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdfe0a-b35a-4101-bfab-a75a140a6baa admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:27.356530 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdfe0a-b35a-4101-bfab-a75a140a6baa admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:27.356806 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdfe0a-b35a-4101-bfab-a75a140a6baa admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:27.357498 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdfe0a-b35a-4101-bfab-a75a140a6baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:27.359555 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdfe0a-b35a-4101-bfab-a75a140a6baa admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:27.385808 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1416/2858] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:03:27 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:27.390352 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff214e1f-6783-44ea-a0ff-bcd5eeb1c76f None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:27.401824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff214e1f-6783-44ea-a0ff-bcd5eeb1c76f None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:27.464862 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12558f-9cbe-4ac5-ae1a-7b3add72a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:27.465538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12558f-9cbe-4ac5-ae1a-7b3add72a553 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:27.465823 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12558f-9cbe-4ac5-ae1a-7b3add72a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:27.466176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12558f-9cbe-4ac5-ae1a-7b3add72a553 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:27.466916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12558f-9cbe-4ac5-ae1a-7b3add72a553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:27.469562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12558f-9cbe-4ac5-ae1a-7b3add72a553 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:27.494769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1443/2859] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:03:27 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 16:03:27.496535 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdfe0a-b35a-4101-bfab-a75a140a6baa admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:27.512425 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdfe0a-b35a-4101-bfab-a75a140a6baa admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:27.523838 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04dbe9-0342-4e12-baee-9ed4333ce224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:27.525475 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04dbe9-0342-4e12-baee-9ed4333ce224 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:27.525722 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04dbe9-0342-4e12-baee-9ed4333ce224 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:27.525937 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04dbe9-0342-4e12-baee-9ed4333ce224 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:27.526762 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04dbe9-0342-4e12-baee-9ed4333ce224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:27.528374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04dbe9-0342-4e12-baee-9ed4333ce224 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:27.545426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1417/2860] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:27.551980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12558f-9cbe-4ac5-ae1a-7b3add72a553 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:27.563608 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12558f-9cbe-4ac5-ae1a-7b3add72a553 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:27.576220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac842c-2aa9-4873-ab64-dbd212c3e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:27.577690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac842c-2aa9-4873-ab64-dbd212c3e6bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:27.577975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac842c-2aa9-4873-ab64-dbd212c3e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:27.578254 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac842c-2aa9-4873-ab64-dbd212c3e6bb admin admin] PATH_INFO: `/v3/projects/b5319537754148f88a7cc73a75f8e013/users/e74ef043abb44849b4831af639e33073/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:27.596261 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac842c-2aa9-4873-ab64-dbd212c3e6bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5319537754148f88a7cc73a75f8e013, user_id=e74ef043abb44849b4831af639e33073, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:27.598972 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac842c-2aa9-4873-ab64-dbd212c3e6bb admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:27.613729 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1444/2861] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:03:27 2022] PUT /identity/v3/projects/b5319537754148f88a7cc73a75f8e013/users/e74ef043abb44849b4831af639e33073/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:27.617903 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04dbe9-0342-4e12-baee-9ed4333ce224 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:27.629381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04dbe9-0342-4e12-baee-9ed4333ce224 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:27.693014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66faf61-b8c2-470f-8254-a47505edc405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:27.693690 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66faf61-b8c2-470f-8254-a47505edc405 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:27.693981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66faf61-b8c2-470f-8254-a47505edc405 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:27.694312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66faf61-b8c2-470f-8254-a47505edc405 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:27.695018 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66faf61-b8c2-470f-8254-a47505edc405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:27.696610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66faf61-b8c2-470f-8254-a47505edc405 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:27.710586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1418/2862] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:27.711953 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac842c-2aa9-4873-ab64-dbd212c3e6bb admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:27.722857 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac842c-2aa9-4873-ab64-dbd212c3e6bb admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:27.727601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925fea1-2f18-4b30-8b18-c25330b1d97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:27.728497 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925fea1-2f18-4b30-8b18-c25330b1d97c admin admin] REQUEST_METHOD: `PUT` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:27.728781 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925fea1-2f18-4b30-8b18-c25330b1d97c admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:27.729058 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925fea1-2f18-4b30-8b18-c25330b1d97c admin admin] PATH_INFO: `/v3/projects/b5319537754148f88a7cc73a75f8e013/users/e74ef043abb44849b4831af639e33073/roles/433478cc318a467c931f13a757f161fa` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:27.731109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925fea1-2f18-4b30-8b18-c25330b1d97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5319537754148f88a7cc73a75f8e013, user_id=e74ef043abb44849b4831af639e33073, role_id=433478cc318a467c931f13a757f161fa)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:27.733789 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925fea1-2f18-4b30-8b18-c25330b1d97c admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:27.747731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1445/2863] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:03:27 2022] PUT /identity/v3/projects/b5319537754148f88a7cc73a75f8e013/users/e74ef043abb44849b4831af639e33073/roles/433478cc318a467c931f13a757f161fa => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:32.549312 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b205c8-9a07-4ed5-9174-87db6172c5b0 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:32.549767 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b205c8-9a07-4ed5-9174-87db6172c5b0 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:32.549851 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b205c8-9a07-4ed5-9174-87db6172c5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:32.567549 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925fea1-2f18-4b30-8b18-c25330b1d97c admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:32.569362 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6925fea1-2f18-4b30-8b18-c25330b1d97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:32.581434 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-83b205c8-9a07-4ed5-9174-87db6172c5b0 None None] MFA Rules not processed for user `e74ef043abb44849b4831af639e33073`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:32.582570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-83b205c8-9a07-4ed5-9174-87db6172c5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:32.645071 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925fea1-2f18-4b30-8b18-c25330b1d97c admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:32.650507 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:32.651428 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:32.651711 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:32.651990 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] PATH_INFO: `/v3/users/4b8d636a0ff145188a69b1393906cf89` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:32.653175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b8d636a0ff145188a69b1393906cf89)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:32.655175 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:32.664231 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1419/2864] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:32.671927 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:32.683426 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b205c8-9a07-4ed5-9174-87db6172c5b0 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:32.685565 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-83b205c8-9a07-4ed5-9174-87db6172c5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:32.701829 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:32.713318 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b8d636a0ff145188a69b1393906cf89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:32.722925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b8d636a0ff145188a69b1393906cf89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:32.734303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b8d636a0ff145188a69b1393906cf89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:32.735215 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b8d636a0ff145188a69b1393906cf89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:32.756270 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-2ad865b3-aff0-4540-bbe2-943fd94f69f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b8d636a0ff145188a69b1393906cf89'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:32.765864 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1446/2865] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:03:32 2022] DELETE /identity/v3/users/4b8d636a0ff145188a69b1393906cf89 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:03:32.768206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62fcf5-513e-40e3-8b02-47d325334ed8 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:32.768206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62fcf5-513e-40e3-8b02-47d325334ed8 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:32.768206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62fcf5-513e-40e3-8b02-47d325334ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:32.773643 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b205c8-9a07-4ed5-9174-87db6172c5b0 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:32.787161 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-ea62fcf5-513e-40e3-8b02-47d325334ed8 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:32.788546 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ea62fcf5-513e-40e3-8b02-47d325334ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:32.842479 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:32.843151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:32.843507 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:32.843855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:32.846033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:32.864140 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1447/2866] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:32.877764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62fcf5-513e-40e3-8b02-47d325334ed8 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:32.877764 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-ea62fcf5-513e-40e3-8b02-47d325334ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:32.940085 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:32.942821 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:32.955523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1420/2867] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:03:32.962290 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62fcf5-513e-40e3-8b02-47d325334ed8 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:32.966054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d110c8d-764e-4902-9906-e6f030c01162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:32.966720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d110c8d-764e-4902-9906-e6f030c01162 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:32.967104 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d110c8d-764e-4902-9906-e6f030c01162 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:32.967906 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d110c8d-764e-4902-9906-e6f030c01162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:32.978896 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d110c8d-764e-4902-9906-e6f030c01162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:32.980475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d110c8d-764e-4902-9906-e6f030c01162 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:32.995494 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1448/2868] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:03:33.401014 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:33.402691 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:33.420852 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d8dd3-92d9-493e-b951-576dc049cf42 service nova] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:33.425944 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:33.426878 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:33.427289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:33.427455 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] PATH_INFO: `/v3/projects/3149517a9f0d4f528aac091dce32b615` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:33.428673 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3149517a9f0d4f528aac091dce32b615)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:33.431417 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:33.493291 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:33.502156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3149517a9f0d4f528aac091dce32b615', 'request_id': 'req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:33.515814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3149517a9f0d4f528aac091dce32b615', 'request_id': 'req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:33.524978 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3149517a9f0d4f528aac091dce32b615', 'request_id': 'req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:33.540551 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] The token cache is being invalidate because project 3149517a9f0d4f528aac091dce32b615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:03:33.540840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:33.541738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-6f6c86b3-1cc0-41fa-8c2f-bb20c174c303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:33.543522 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1421/2869] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:03:33 2022] DELETE /identity/v3/projects/3149517a9f0d4f528aac091dce32b615 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:33.549457 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268521f-3f70-46e3-9063-42ccc98f1352 None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:33.549830 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268521f-3f70-46e3-9063-42ccc98f1352 None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:33.550176 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268521f-3f70-46e3-9063-42ccc98f1352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:33.584994 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-7268521f-3f70-46e3-9063-42ccc98f1352 None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:33.586187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7268521f-3f70-46e3-9063-42ccc98f1352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:33.618134 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:33.651605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1449/2870] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:33.661267 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79989ec-7059-4878-a81e-16ba9559c10c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:33.661795 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79989ec-7059-4878-a81e-16ba9559c10c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:33.662069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79989ec-7059-4878-a81e-16ba9559c10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:33.683220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-e79989ec-7059-4878-a81e-16ba9559c10c None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:33.684441 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-e79989ec-7059-4878-a81e-16ba9559c10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:33.693157 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1422/2871] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:33.697672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268521f-3f70-46e3-9063-42ccc98f1352 None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:33.713169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268521f-3f70-46e3-9063-42ccc98f1352 None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:33.719024 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827193d-f224-4f3c-9fd5-fdadbb760530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:33.720086 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827193d-f224-4f3c-9fd5-fdadbb760530 admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:33.720576 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827193d-f224-4f3c-9fd5-fdadbb760530 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:33.721084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827193d-f224-4f3c-9fd5-fdadbb760530 admin admin] PATH_INFO: `/v3/domains` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:33.722109 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827193d-f224-4f3c-9fd5-fdadbb760530 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86640) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 16:03:33.722613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827193d-f224-4f3c-9fd5-fdadbb760530 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:33.725054 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827193d-f224-4f3c-9fd5-fdadbb760530 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:33.744010 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1450/2872] 10.222.0.62 () {58 vars in 1152 bytes} [Wed Jun 15 16:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 16:03:33.744909 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79989ec-7059-4878-a81e-16ba9559c10c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:33.755007 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79989ec-7059-4878-a81e-16ba9559c10c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:33.759897 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723e41d-307e-40b8-9ecb-00e87cd49f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:33.760652 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723e41d-307e-40b8-9ecb-00e87cd49f37 admin admin] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:33.761019 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723e41d-307e-40b8-9ecb-00e87cd49f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:33.761354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723e41d-307e-40b8-9ecb-00e87cd49f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:33.762122 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723e41d-307e-40b8-9ecb-00e87cd49f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:33.764411 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723e41d-307e-40b8-9ecb-00e87cd49f37 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:33.794765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1423/2873] 10.222.0.62 () {60 vars in 1152 bytes} [Wed Jun 15 16:03:33 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:33.796824 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827193d-f224-4f3c-9fd5-fdadbb760530 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:33.807922 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827193d-f224-4f3c-9fd5-fdadbb760530 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:33.879309 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10280-1ebe-45b6-a7e0-6a6d8f007b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:33.880038 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10280-1ebe-45b6-a7e0-6a6d8f007b88 admin admin] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:33.880316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10280-1ebe-45b6-a7e0-6a6d8f007b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:33.880589 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10280-1ebe-45b6-a7e0-6a6d8f007b88 admin admin] PATH_INFO: `/v3/users` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:33.881305 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10280-1ebe-45b6-a7e0-6a6d8f007b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:33.883271 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10280-1ebe-45b6-a7e0-6a6d8f007b88 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:33.903822 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1451/2874] 10.222.0.62 () {60 vars in 1143 bytes} [Wed Jun 15 16:03:33 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 16:03:33.913005 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723e41d-307e-40b8-9ecb-00e87cd49f37 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:33.924523 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723e41d-307e-40b8-9ecb-00e87cd49f37 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:33.930140 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557f5b-3520-465c-bf70-fb9191f87982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:33.930839 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557f5b-3520-465c-bf70-fb9191f87982 admin admin] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:33.931123 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557f5b-3520-465c-bf70-fb9191f87982 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:33.931405 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557f5b-3520-465c-bf70-fb9191f87982 admin admin] PATH_INFO: `/v3/roles` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:33.932137 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab557f5b-3520-465c-bf70-fb9191f87982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:33.933975 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab557f5b-3520-465c-bf70-fb9191f87982 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:33.945610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1424/2875] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:33.949389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10280-1ebe-45b6-a7e0-6a6d8f007b88 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:33.959766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10280-1ebe-45b6-a7e0-6a6d8f007b88 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:33.966699 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519bab77-e1d9-49ec-835a-9b238f7f0f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:33.967926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bab77-e1d9-49ec-835a-9b238f7f0f4a admin admin] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:33.968223 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bab77-e1d9-49ec-835a-9b238f7f0f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:33.968557 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bab77-e1d9-49ec-835a-9b238f7f0f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:33.969296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519bab77-e1d9-49ec-835a-9b238f7f0f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:33.970937 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519bab77-e1d9-49ec-835a-9b238f7f0f4a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:33.997854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1452/2876] 10.222.0.62 () {58 vars in 1121 bytes} [Wed Jun 15 16:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 16:03:34.001888 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557f5b-3520-465c-bf70-fb9191f87982 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:34.017867 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557f5b-3520-465c-bf70-fb9191f87982 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:34.024264 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e723261-459c-4a14-925e-659eb66d24f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:34.025288 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e723261-459c-4a14-925e-659eb66d24f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:34.025638 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e723261-459c-4a14-925e-659eb66d24f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:34.026037 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e723261-459c-4a14-925e-659eb66d24f3 admin admin] PATH_INFO: `/v3/projects/f6fa91931766406aaabd1e99bc2d7409/users/98ba0905f1a84e9db990ab213e6aabf7/roles/4d4438a26f504e9f84ba62dc395f926e` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:34.057659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e723261-459c-4a14-925e-659eb66d24f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fa91931766406aaabd1e99bc2d7409, user_id=98ba0905f1a84e9db990ab213e6aabf7, role_id=4d4438a26f504e9f84ba62dc395f926e)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:34.060446 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e723261-459c-4a14-925e-659eb66d24f3 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:34.075110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1425/2877] 10.222.0.62 () {60 vars in 1482 bytes} [Wed Jun 15 16:03:34 2022] PUT /identity/v3/projects/f6fa91931766406aaabd1e99bc2d7409/users/98ba0905f1a84e9db990ab213e6aabf7/roles/4d4438a26f504e9f84ba62dc395f926e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:03:34.087705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26d629-00c2-42ae-b464-7d261aaf61da None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:34.088133 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26d629-00c2-42ae-b464-7d261aaf61da None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:34.088540 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26d629-00c2-42ae-b464-7d261aaf61da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:34.120883 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-5c26d629-00c2-42ae-b464-7d261aaf61da None None] MFA Rules not processed for user `98ba0905f1a84e9db990ab213e6aabf7`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:34.122466 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5c26d629-00c2-42ae-b464-7d261aaf61da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:34.140404 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6011fc-f2dd-4789-a5fc-601a306e3a40 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:34.140704 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6011fc-f2dd-4789-a5fc-601a306e3a40 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:34.140988 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6011fc-f2dd-4789-a5fc-601a306e3a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:34.162932 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-7a6011fc-f2dd-4789-a5fc-601a306e3a40 None None] MFA Rules not processed for user `34e42d4cebe14642a71209ddd2b25fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:34.164309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7a6011fc-f2dd-4789-a5fc-601a306e3a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:34.209200 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1453/2878] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:34.218153 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26d629-00c2-42ae-b464-7d261aaf61da None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:34.219962 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5c26d629-00c2-42ae-b464-7d261aaf61da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:34.248987 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1426/2879] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 16:03:34.264869 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:34.311998 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26d629-00c2-42ae-b464-7d261aaf61da None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:34.317454 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d594d83-44b1-4404-8a19-fc430bd01d80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:34.318601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d594d83-44b1-4404-8a19-fc430bd01d80 service cinder] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:34.319230 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d594d83-44b1-4404-8a19-fc430bd01d80 service cinder] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:34.319705 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d594d83-44b1-4404-8a19-fc430bd01d80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:34.332287 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d594d83-44b1-4404-8a19-fc430bd01d80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:34.334339 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d594d83-44b1-4404-8a19-fc430bd01d80 service cinder] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:34.350335 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1454/2880] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:03:37.007207 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6011fc-f2dd-4789-a5fc-601a306e3a40 None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:37.075629 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6011fc-f2dd-4789-a5fc-601a306e3a40 None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:37.084675 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:37.085636 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:37.086195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:37.086195 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] PATH_INFO: `/v3/users/98ba0905f1a84e9db990ab213e6aabf7` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:37.087488 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ba0905f1a84e9db990ab213e6aabf7)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:37.089439 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:37.102840 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:37.148342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ba0905f1a84e9db990ab213e6aabf7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.157214 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ba0905f1a84e9db990ab213e6aabf7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.165093 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ba0905f1a84e9db990ab213e6aabf7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.166131 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ba0905f1a84e9db990ab213e6aabf7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.166961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ba0905f1a84e9db990ab213e6aabf7'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.187342 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1427/2881] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:03:37 2022] DELETE /identity/v3/users/98ba0905f1a84e9db990ab213e6aabf7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:37.191870 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03502873-df95-4159-9fd2-954a1a788a7b None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:37.192391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03502873-df95-4159-9fd2-954a1a788a7b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:37.192512 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03502873-df95-4159-9fd2-954a1a788a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:37.211867 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-03502873-df95-4159-9fd2-954a1a788a7b None None] MFA Rules not processed for user `38729f2fb8864984b35336009a142811`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:03:37.213098 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-03502873-df95-4159-9fd2-954a1a788a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:37.280950 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1455/2882] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:03:37.291190 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:37.292631 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:37.352385 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb5ab3-5806-48b4-b85c-600b0339cb89 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:37.356035 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3b76f-2a2d-4d37-ab6a-ff73d16ae3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:37.356765 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3b76f-2a2d-4d37-ab6a-ff73d16ae3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:37.356954 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3b76f-2a2d-4d37-ab6a-ff73d16ae3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:37.357230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3b76f-2a2d-4d37-ab6a-ff73d16ae3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:37.369369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3b76f-2a2d-4d37-ab6a-ff73d16ae3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:37.373622 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3b76f-2a2d-4d37-ab6a-ff73d16ae3e0 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:37.384890 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1428/2883] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:03:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:03:37.666980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03502873-df95-4159-9fd2-954a1a788a7b None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:03:37.668658 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-03502873-df95-4159-9fd2-954a1a788a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:03:37.682478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03502873-df95-4159-9fd2-954a1a788a7b None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:03:37.687129 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:03:37.687989 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:03:37.688266 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:03:37.688539 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] PATH_INFO: `/v3/projects/f6fa91931766406aaabd1e99bc2d7409` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:03:37.690235 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6fa91931766406aaabd1e99bc2d7409)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:03:37.692393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:03:37.740391 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6fa91931766406aaabd1e99bc2d7409', 'request_id': 'req-d53309fe-53fa-4054-9608-68c132562980'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.746632 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6fa91931766406aaabd1e99bc2d7409', 'request_id': 'req-d53309fe-53fa-4054-9608-68c132562980'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.753946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6fa91931766406aaabd1e99bc2d7409', 'request_id': 'req-d53309fe-53fa-4054-9608-68c132562980'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.764521 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] The token cache is being invalidate because project f6fa91931766406aaabd1e99bc2d7409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:03:37.764806 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.765477 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:03:37.767051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1456/2884] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:03:37 2022] DELETE /identity/v3/projects/f6fa91931766406aaabd1e99bc2d7409 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:03:47.379791 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=48,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:03:47.761324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:04:00.107756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3b76f-2a2d-4d37-ab6a-ff73d16ae3e0 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:04:00.109180 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d3c3b76f-2a2d-4d37-ab6a-ff73d16ae3e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:04:00.197504 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3b76f-2a2d-4d37-ab6a-ff73d16ae3e0 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:04:00.256111 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2ccf4-bd10-4dfc-b68b-5b5faa172bed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:04:00.256761 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2ccf4-bd10-4dfc-b68b-5b5faa172bed service placement] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:04:00.257046 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2ccf4-bd10-4dfc-b68b-5b5faa172bed service placement] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:04:00.257325 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2ccf4-bd10-4dfc-b68b-5b5faa172bed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:04:00.259225 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-13a2ccf4-bd10-4dfc-b68b-5b5faa172bed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:04:00.270963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2ccf4-bd10-4dfc-b68b-5b5faa172bed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:04:00.272531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2ccf4-bd10-4dfc-b68b-5b5faa172bed service placement] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:04:00.283039 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1429/2885] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:04:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:04:10.278514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=67 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:04:16.208732 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:04:16.210220 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:04:16.266446 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53309fe-53fa-4054-9608-68c132562980 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:04:16.270084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d2e43-807e-4826-8e55-0af7f28afd5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:04:16.270773 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d2e43-807e-4826-8e55-0af7f28afd5f service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:04:16.271064 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d2e43-807e-4826-8e55-0af7f28afd5f service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:04:16.271344 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d2e43-807e-4826-8e55-0af7f28afd5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:04:16.273107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f51d2e43-807e-4826-8e55-0af7f28afd5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:04:16.334422 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d2e43-807e-4826-8e55-0af7f28afd5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:04:16.335904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d2e43-807e-4826-8e55-0af7f28afd5f service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:04:16.345616 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1457/2886] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:04:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:04:26.341152 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=50 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:05:23.324760 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2ccf4-bd10-4dfc-b68b-5b5faa172bed service placement] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:05:23.334104 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2ccf4-bd10-4dfc-b68b-5b5faa172bed service placement] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:05:23.338375 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bfdd8-1e99-4527-845c-403d4fa4bbc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:05:23.339042 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bfdd8-1e99-4527-845c-403d4fa4bbc8 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:05:23.339326 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bfdd8-1e99-4527-845c-403d4fa4bbc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:05:23.339605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bfdd8-1e99-4527-845c-403d4fa4bbc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:05:23.341374 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-3c6bfdd8-1e99-4527-845c-403d4fa4bbc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:05:23.430770 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bfdd8-1e99-4527-845c-403d4fa4bbc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:05:23.432177 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bfdd8-1e99-4527-845c-403d4fa4bbc8 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:05:23.453266 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1430/2887] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:05:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:05:24.627013 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d2e43-807e-4826-8e55-0af7f28afd5f service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:05:24.636210 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d2e43-807e-4826-8e55-0af7f28afd5f service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:05:24.639854 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:05:24.640482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:05:24.640762 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:05:24.641037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:05:24.642814 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:05:24.656562 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:05:24.657975 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:05:24.669051 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1458/2888] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:05:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:05:28.120721 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26385ae5-3d13-4912-a083-6f96bc408ce8 None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:05:28.121181 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26385ae5-3d13-4912-a083-6f96bc408ce8 None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:05:28.121290 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26385ae5-3d13-4912-a083-6f96bc408ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:05:28.140370 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-26385ae5-3d13-4912-a083-6f96bc408ce8 None None] MFA Rules not processed for user `f18270589e8745a286e3c78a7f86b25d`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:05:28.141568 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-26385ae5-3d13-4912-a083-6f96bc408ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:05:28.194835 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1431/2889] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:05:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:05:30.584506 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:05:30.585971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:05:30.597603 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647c07-dd3c-471e-8b56-1c7a3879498f service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:05:30.602449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:05:30.603226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:05:30.603607 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:05:30.603956 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:05:30.605909 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:05:30.616595 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:05:30.619388 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:05:30.631218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1459/2890] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:05:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:05:38.188293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=71 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:05:40.625876 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=12 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:06:01.866004 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15845f3-62dc-4bd2-8c6b-bf608a45bb3c None None] REQUEST_METHOD: `POST` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:01.866497 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15845f3-62dc-4bd2-8c6b-bf608a45bb3c None None] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:01.866613 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15845f3-62dc-4bd2-8c6b-bf608a45bb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:01.922981 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.auth.core [None req-c15845f3-62dc-4bd2-8c6b-bf608a45bb3c None None] MFA Rules not processed for user `2f73ecf1501a4ab181ad9a24ac54aec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:06:01.924166 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c15845f3-62dc-4bd2-8c6b-bf608a45bb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:01.986377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1432/2891] 10.222.0.62 () {58 vars in 967 bytes} [Wed Jun 15 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:06:02.026296 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:02.027748 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:02.116731 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92da50-5e3f-4b16-9098-abab397b53ba service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:02.121707 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f1674-0f93-4329-a1cb-f83f3b645b46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e3bb6553264387936e9a534afc3ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:02.122340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f1674-0f93-4329-a1cb-f83f3b645b46 service swift] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:02.122621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f1674-0f93-4329-a1cb-f83f3b645b46 service swift] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:02.122926 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f1674-0f93-4329-a1cb-f83f3b645b46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:02.130875 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f1674-0f93-4329-a1cb-f83f3b645b46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:06:02.132475 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f1674-0f93-4329-a1cb-f83f3b645b46 service swift] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:06:02.140449 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1460/2892] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Jun 15 16:06:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 16:06:05.254920 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15845f3-62dc-4bd2-8c6b-bf608a45bb3c None None] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:05.256369 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c15845f3-62dc-4bd2-8c6b-bf608a45bb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:05.272489 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15845f3-62dc-4bd2-8c6b-bf608a45bb3c None None] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:05.278508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:05.279235 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:05.279514 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:05.279792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:05.281709 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:05.295150 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:06:05.296774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:06:05.309199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1433/2893] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:06:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:06:08.397744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f1674-0f93-4329-a1cb-f83f3b645b46 service swift] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:08.417809 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f1674-0f93-4329-a1cb-f83f3b645b46 service swift] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:08.426690 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3e83f-e7c9-48bd-9195-ea8a25e6baf3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:08.427851 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3e83f-e7c9-48bd-9195-ea8a25e6baf3 service placement] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:08.428126 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3e83f-e7c9-48bd-9195-ea8a25e6baf3 service placement] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:08.428404 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3e83f-e7c9-48bd-9195-ea8a25e6baf3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:08.432355 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-9fe3e83f-e7c9-48bd-9195-ea8a25e6baf3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:08.454046 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3e83f-e7c9-48bd-9195-ea8a25e6baf3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:06:08.459419 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3e83f-e7c9-48bd-9195-ea8a25e6baf3 service placement] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:06:08.479695 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1461/2894] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:06:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:06:08.487454 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:08.488912 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:08.568468 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e4c2f-beec-4c4d-83d7-50f5f26c9e75 service placement] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:08.572993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:08.573862 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:08.574147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:08.574444 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] PATH_INFO: `/v3/users/12a0715ba0494afd8194655dee4b80ab` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:08.575656 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a0715ba0494afd8194655dee4b80ab)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:06:08.577568 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:06:08.623764 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a0715ba0494afd8194655dee4b80ab'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.627347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:06:08.631961 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a0715ba0494afd8194655dee4b80ab'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.640347 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a0715ba0494afd8194655dee4b80ab'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.641049 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a0715ba0494afd8194655dee4b80ab'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.641659 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a0715ba0494afd8194655dee4b80ab'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.659187 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1434/2895] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:06:08 2022] DELETE /identity/v3/users/12a0715ba0494afd8194655dee4b80ab => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:06:08.828005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3e83f-e7c9-48bd-9195-ea8a25e6baf3 service placement] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:08.843303 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3e83f-e7c9-48bd-9195-ea8a25e6baf3 service placement] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:08.899307 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:08.900206 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:08.900487 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:08.900763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] PATH_INFO: `/v3/projects/4cd1cf864eca4b0eb909e812b7de3f3f` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:08.901957 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd1cf864eca4b0eb909e812b7de3f3f)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:06:08.903920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:06:08.950169 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd1cf864eca4b0eb909e812b7de3f3f', 'request_id': 'req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.956067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd1cf864eca4b0eb909e812b7de3f3f', 'request_id': 'req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.959591 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:06:08.963920 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd1cf864eca4b0eb909e812b7de3f3f', 'request_id': 'req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.975316 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] The token cache is being invalidate because project 4cd1cf864eca4b0eb909e812b7de3f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:06:08.975601 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.976219 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:08.978249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1462/2896] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:06:08 2022] DELETE /identity/v3/projects/4cd1cf864eca4b0eb909e812b7de3f3f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:06:08.982186 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:08.983669 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:08.997301 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7e194-65d0-48c5-ad8b-9fbce0007300 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:09.052926 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:09.053794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:09.054081 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:09.054381 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] PATH_INFO: `/v3/users/959127669a8f48198074d457f4384521` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:09.056189 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=959127669a8f48198074d457f4384521)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:06:09.058169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:06:09.101717 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '959127669a8f48198074d457f4384521'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.110744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '959127669a8f48198074d457f4384521'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.118334 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '959127669a8f48198074d457f4384521'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.118993 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '959127669a8f48198074d457f4384521'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.119610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '959127669a8f48198074d457f4384521'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.136428 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1435/2897] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:06:08 2022] DELETE /identity/v3/users/959127669a8f48198074d457f4384521 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:06:09.309912 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:09.311460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:09.324538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fc7c2-ee05-4f6b-bbc2-f66dac5fb521 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:09.382395 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:09.383300 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:09.383572 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:09.383847 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] PATH_INFO: `/v3/projects/64df054a70004df3b8cc63acaedd0c2e` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:09.385032 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=64df054a70004df3b8cc63acaedd0c2e)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:06:09.387127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:06:09.433265 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64df054a70004df3b8cc63acaedd0c2e', 'request_id': 'req-3e3cff05-9ab1-4950-813b-98783dfc47cf'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.439585 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64df054a70004df3b8cc63acaedd0c2e', 'request_id': 'req-3e3cff05-9ab1-4950-813b-98783dfc47cf'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.447393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64df054a70004df3b8cc63acaedd0c2e', 'request_id': 'req-3e3cff05-9ab1-4950-813b-98783dfc47cf'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.458137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] The token cache is being invalidate because project 64df054a70004df3b8cc63acaedd0c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:06:09.458445 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.459137 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:06:09.460718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1463/2898] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:06:09 2022] DELETE /identity/v3/projects/64df054a70004df3b8cc63acaedd0c2e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:06:19.132959 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=65,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:06:19.455009 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=55,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:06:59.531615 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:59.533220 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:59.592525 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3d89a-6fa5-4e09-880b-95017ed4ddd0 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:59.643509 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:59.644170 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:59.644457 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:59.644744 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:59.646559 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:59.659106 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:06:59.660548 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:06:59.669351 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1436/2899] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:06:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:06:59.751185 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:59.752662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:59.765260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cff05-9ab1-4950-813b-98783dfc47cf admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:59.768769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:59.769431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:59.769712 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:59.769988 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:59.771718 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:59.841121 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:06:59.842551 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:06:59.853360 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1464/2900] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:06:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:06:59.913345 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:06:59.914792 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:06:59.982484 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad3daf-ec8e-4239-81b5-fc2f25e7341c service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:06:59.987508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:06:59.988143 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:06:59.988429 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:06:59.988708 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:06:59.991077 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:07:00.002069 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:07:00.003720 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:07:00.014134 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1437/2901] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:06:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:07:09.848660 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=34 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:07:10.009202 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=78 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:07:12.466853 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:07:12.469460 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:07:12.487042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2bdcc-5f02-45ab-9898-f446c3d0b129 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:07:12.493564 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daba87c-236c-4faa-a822-17aec49f73f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:07:12.494218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daba87c-236c-4faa-a822-17aec49f73f5 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:07:12.494504 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daba87c-236c-4faa-a822-17aec49f73f5 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:07:12.494813 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daba87c-236c-4faa-a822-17aec49f73f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:07:12.496804 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-5daba87c-236c-4faa-a822-17aec49f73f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:07:12.510043 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daba87c-236c-4faa-a822-17aec49f73f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:07:12.511674 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daba87c-236c-4faa-a822-17aec49f73f5 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:07:12.522716 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1465/2902] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:07:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:07:22.517877 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=6 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:08:02.999473 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:08:03.000885 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:03.056910 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50387b-6536-43f6-ae20-9866aa0794a1 service placement] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:08:03.060655 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d5bc-7724-429d-9b71-e50f7e6148a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bd66c1b7c7147a281ef2e7253224069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:08:03.061289 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d5bc-7724-429d-9b71-e50f7e6148a4 service cinder] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:08:03.061570 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d5bc-7724-429d-9b71-e50f7e6148a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:08:03.061849 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d5bc-7724-429d-9b71-e50f7e6148a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:08:03.063635 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-92d0d5bc-7724-429d-9b71-e50f7e6148a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:03.120871 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0d5bc-7724-429d-9b71-e50f7e6148a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:08:03.122308 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0d5bc-7724-429d-9b71-e50f7e6148a4 service cinder] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:08:03.131536 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1438/2903] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:08:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:08:13.127293 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=48 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:08:15.141441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90b68e-1780-456d-8686-405f21cf6c4b None None] REQUEST_METHOD: `POST` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:08:15.141967 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90b68e-1780-456d-8686-405f21cf6c4b None None] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:08:15.142075 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90b68e-1780-456d-8686-405f21cf6c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:08:15.159005 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.auth.core [None req-aa90b68e-1780-456d-8686-405f21cf6c4b None None] MFA Rules not processed for user `f18270589e8745a286e3c78a7f86b25d`. Rule list: `[]` (Enabled: `True`). {{(pid=86640) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 16:08:15.160218 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-aa90b68e-1780-456d-8686-405f21cf6c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:15.167172 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1466/2904] 10.222.0.62 () {58 vars in 957 bytes} [Wed Jun 15 16:08:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 16:08:15.481223 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d5bc-7724-429d-9b71-e50f7e6148a4 service cinder] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:08:15.491212 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d5bc-7724-429d-9b71-e50f7e6148a4 service cinder] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:08:15.494812 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b4599-de88-4688-9966-0d159a4a8568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:08:15.495450 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b4599-de88-4688-9966-0d159a4a8568 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:08:15.495738 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b4599-de88-4688-9966-0d159a4a8568 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:08:15.496021 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b4599-de88-4688-9966-0d159a4a8568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:08:15.497807 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-856b4599-de88-4688-9966-0d159a4a8568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:15.507364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b4599-de88-4688-9966-0d159a4a8568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:08:15.508787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b4599-de88-4688-9966-0d159a4a8568 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:08:15.517159 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1439/2905] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:08:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:08:22.319916 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90b68e-1780-456d-8686-405f21cf6c4b None None] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:08:22.352676 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90b68e-1780-456d-8686-405f21cf6c4b None None] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:08:22.359224 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e421a-56a8-458a-bd0a-feaf3808818e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:08:22.360534 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e421a-56a8-458a-bd0a-feaf3808818e service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:08:22.361067 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e421a-56a8-458a-bd0a-feaf3808818e service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:08:22.361431 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e421a-56a8-458a-bd0a-feaf3808818e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:08:22.363769 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-273e421a-56a8-458a-bd0a-feaf3808818e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:22.378127 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e421a-56a8-458a-bd0a-feaf3808818e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:08:22.379751 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e421a-56a8-458a-bd0a-feaf3808818e service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:08:22.394181 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1467/2906] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:08:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:08:25.514117 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=6 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:08:32.386249 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=9 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:08:37.000354 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b4599-de88-4688-9966-0d159a4a8568 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:08:37.009068 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b4599-de88-4688-9966-0d159a4a8568 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:08:37.012676 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30f613-c9fc-470c-85be-fc5435a0dd7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:08:37.013310 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30f613-c9fc-470c-85be-fc5435a0dd7e service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:08:37.013605 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30f613-c9fc-470c-85be-fc5435a0dd7e service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:08:37.013889 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30f613-c9fc-470c-85be-fc5435a0dd7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:08:37.015671 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-ef30f613-c9fc-470c-85be-fc5435a0dd7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:37.080056 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30f613-c9fc-470c-85be-fc5435a0dd7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:08:37.081485 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30f613-c9fc-470c-85be-fc5435a0dd7e service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:08:37.091032 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1440/2907] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:08:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:08:38.618980 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e421a-56a8-458a-bd0a-feaf3808818e service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:08:38.630766 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e421a-56a8-458a-bd0a-feaf3808818e service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:08:38.636681 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4327e4eed5a1436ea1a00c2209bc5750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:08:38.637324 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:08:38.637642 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:08:38.637973 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:08:38.639900 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:38.653295 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:08:38.654924 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:08:38.665892 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1468/2908] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:08:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:08:47.086641 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=30 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:08:48.661306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=6 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:08:49.338710 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30f613-c9fc-470c-85be-fc5435a0dd7e service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:08:49.347610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30f613-c9fc-470c-85be-fc5435a0dd7e service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:08:49.351205 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:08:49.351841 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:08:49.352127 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:08:49.352406 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:08:49.354183 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:49.412756 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:08:49.414169 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:08:49.423646 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1441/2909] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:08:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:08:55.238260 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:08:55.239722 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:55.251280 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f0e45-bac1-4309-9982-0bfffed61ae3 service nova] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:08:55.255461 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efbe04-d080-4c86-abae-467d097240b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:08:55.256094 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efbe04-d080-4c86-abae-467d097240b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:08:55.256375 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efbe04-d080-4c86-abae-467d097240b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:08:55.256647 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efbe04-d080-4c86-abae-467d097240b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:08:55.258349 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-f1efbe04-d080-4c86-abae-467d097240b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:08:55.320904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efbe04-d080-4c86-abae-467d097240b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:08:55.322272 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efbe04-d080-4c86-abae-467d097240b0 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:08:55.331730 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1469/2910] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 16:08:59.418931 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=28 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:09:01.224262 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:09:01.226492 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:09:01.241028 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a549a41-ddfd-4ec4-836b-6b7c02e52e43 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:09:01.245443 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:09:01.246309 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:09:01.246590 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:09:01.246914 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] PATH_INFO: `/v3/users/34e42d4cebe14642a71209ddd2b25fd2` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:09:01.248110 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e42d4cebe14642a71209ddd2b25fd2)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:09:01.249989 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:09:01.294616 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e42d4cebe14642a71209ddd2b25fd2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.304265 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e42d4cebe14642a71209ddd2b25fd2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.311471 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e42d4cebe14642a71209ddd2b25fd2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.312096 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e42d4cebe14642a71209ddd2b25fd2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.312700 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e42d4cebe14642a71209ddd2b25fd2'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.329846 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1442/2911] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:09:01 2022] DELETE /identity/v3/users/34e42d4cebe14642a71209ddd2b25fd2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:09:01.500187 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efbe04-d080-4c86-abae-467d097240b0 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:09:01.510243 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efbe04-d080-4c86-abae-467d097240b0 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:09:01.568826 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:09:01.569441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:09:01.569723 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:09:01.569997 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] PATH_INFO: `/v3/projects/202b681b581b4f6ea4fe9fcbf049c185` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:09:01.571274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] RBAC: Authorizing `identity:delete_project(project_id=202b681b581b4f6ea4fe9fcbf049c185)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:09:01.573201 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:09:01.620866 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '202b681b581b4f6ea4fe9fcbf049c185', 'request_id': 'req-7968451a-f042-4f46-9a5e-c1c634af600a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.627007 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '202b681b581b4f6ea4fe9fcbf049c185', 'request_id': 'req-7968451a-f042-4f46-9a5e-c1c634af600a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.634116 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '202b681b581b4f6ea4fe9fcbf049c185', 'request_id': 'req-7968451a-f042-4f46-9a5e-c1c634af600a'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.644459 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] The token cache is being invalidate because project 202b681b581b4f6ea4fe9fcbf049c185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:09:01.644744 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.645389 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.646979 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1470/2912] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:09:01 2022] DELETE /identity/v3/projects/202b681b581b4f6ea4fe9fcbf049c185 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 16:09:01.650982 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:09:01.652394 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:09:01.665752 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6b1f2-ce56-4ed2-9c06-d2ee2e23d95b admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:09:01.720751 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:09:01.721610 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:09:01.721892 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:09:01.722167 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] PATH_INFO: `/v3/users/e74ef043abb44849b4831af639e33073` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:09:01.723349 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74ef043abb44849b4831af639e33073)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:09:01.725230 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:09:01.770436 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74ef043abb44849b4831af639e33073'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.777963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74ef043abb44849b4831af639e33073'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.785460 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74ef043abb44849b4831af639e33073'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.786154 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74ef043abb44849b4831af639e33073'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.786787 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74ef043abb44849b4831af639e33073'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:01.794427 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:09:01.806569 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1443/2913] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:09:01 2022] DELETE /identity/v3/users/e74ef043abb44849b4831af639e33073 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:09:01.985533 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:09:01.987743 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:09:02.000946 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968451a-f042-4f46-9a5e-c1c634af600a admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:09:02.022183 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:09:02.061033 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:09:02.061886 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:09:02.062202 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:09:02.062456 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] PATH_INFO: `/v3/projects/b5319537754148f88a7cc73a75f8e013` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:09:02.063672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5319537754148f88a7cc73a75f8e013)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:09:02.065590 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:09:02.110274 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5319537754148f88a7cc73a75f8e013', 'request_id': 'req-d7e4f52c-e140-4dcb-8375-d9976d66578b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:02.116439 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5319537754148f88a7cc73a75f8e013', 'request_id': 'req-d7e4f52c-e140-4dcb-8375-d9976d66578b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:02.124255 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5319537754148f88a7cc73a75f8e013', 'request_id': 'req-d7e4f52c-e140-4dcb-8375-d9976d66578b'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:02.134971 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] The token cache is being invalidate because project b5319537754148f88a7cc73a75f8e013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86640) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:09:02.135252 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:02.135865 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:09:02.137393 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1471/2914] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:09:01 2022] DELETE /identity/v3/projects/b5319537754148f88a7cc73a75f8e013 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:09:11.803640 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:09:12.132107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:09:13.768199 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:09:13.769963 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:09:13.839581 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9e014-ef7d-4b98-bb22-17f48e0d3916 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:09:13.897201 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc48c-4dff-4dd0-a89e-df26b1e64cda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:09:13.897847 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc48c-4dff-4dd0-a89e-df26b1e64cda service placement] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:09:13.898135 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc48c-4dff-4dd0-a89e-df26b1e64cda service placement] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:09:13.898419 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc48c-4dff-4dd0-a89e-df26b1e64cda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:09:13.900239 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-a5bcc48c-4dff-4dd0-a89e-df26b1e64cda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:09:13.911826 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc48c-4dff-4dd0-a89e-df26b1e64cda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:09:13.913407 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc48c-4dff-4dd0-a89e-df26b1e64cda service placement] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:09:13.923531 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1444/2915] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:09:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:09:23.918984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=58 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:10:28.485947 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:10:28.487693 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:10:28.558538 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4f52c-e140-4dcb-8375-d9976d66578b admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:10:28.562629 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:10:28.563325 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:10:28.563613 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:10:28.564042 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:10:28.566904 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:10:28.654170 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:10:28.655965 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:10:28.668727 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1472/2916] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:10:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:10:29.894023 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc48c-4dff-4dd0-a89e-df26b1e64cda service placement] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:10:29.903585 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc48c-4dff-4dd0-a89e-df26b1e64cda service placement] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:10:29.907151 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8108129-e402-4171-bd27-d21b63d2da6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:10:29.907814 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8108129-e402-4171-bd27-d21b63d2da6f service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:10:29.908096 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8108129-e402-4171-bd27-d21b63d2da6f service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:10:29.908377 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8108129-e402-4171-bd27-d21b63d2da6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:10:29.910156 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-c8108129-e402-4171-bd27-d21b63d2da6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:10:29.923749 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8108129-e402-4171-bd27-d21b63d2da6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:10:29.925163 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8108129-e402-4171-bd27-d21b63d2da6f service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:10:29.937774 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1445/2917] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:10:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:10:30.243928 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:10:30.245465 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:10:30.256262 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78164090-74bd-4741-bc58-27b0620127e0 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:10:30.259880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:10:30.260520 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:10:30.260807 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:10:30.261084 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:10:30.262874 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:10:30.272663 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:10:30.274107 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:10:30.282593 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1473/2918] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:10:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:10:39.933026 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=6 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:10:40.278925 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=60 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:10:40.967586 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8108129-e402-4171-bd27-d21b63d2da6f service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:10:40.976415 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8108129-e402-4171-bd27-d21b63d2da6f service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:10:40.980062 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739dda35-e9c6-46ad-bf14-faf0cb5f3ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf1915b9919442d692bcedf25f53320c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:10:40.980697 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739dda35-e9c6-46ad-bf14-faf0cb5f3ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:10:40.980984 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739dda35-e9c6-46ad-bf14-faf0cb5f3ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:10:40.981263 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739dda35-e9c6-46ad-bf14-faf0cb5f3ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:10:40.983088 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-739dda35-e9c6-46ad-bf14-faf0cb5f3ad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:10:41.046571 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739dda35-e9c6-46ad-bf14-faf0cb5f3ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:10:41.048198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739dda35-e9c6-46ad-bf14-faf0cb5f3ad1 service neutron] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:10:41.057797 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1446/2919] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:10:47.269211 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:10:47.270717 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:10:47.285763 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600b0cb-0307-4c00-93a7-6ad2079c75c6 service neutron] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:10:47.290413 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:10:47.291320 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:10:47.291605 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:10:47.291889 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] PATH_INFO: `/v3/users/f18270589e8745a286e3c78a7f86b25d` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:10:47.293092 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f18270589e8745a286e3c78a7f86b25d)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:10:47.295069 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:10:47.342167 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f18270589e8745a286e3c78a7f86b25d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.348580 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f18270589e8745a286e3c78a7f86b25d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.359441 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f18270589e8745a286e3c78a7f86b25d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.360164 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f18270589e8745a286e3c78a7f86b25d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.373652 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f18270589e8745a286e3c78a7f86b25d'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.380329 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1474/2920] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:10:47 2022] DELETE /identity/v3/users/f18270589e8745a286e3c78a7f86b25d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:10:47.571863 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739dda35-e9c6-46ad-bf14-faf0cb5f3ad1 service neutron] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:10:47.582794 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739dda35-e9c6-46ad-bf14-faf0cb5f3ad1 service neutron] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:10:47.639244 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:10:47.640114 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:10:47.640401 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:10:47.640679 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] PATH_INFO: `/v3/projects/31480d73f44447679761a5acac79dc02` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:10:47.641855 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31480d73f44447679761a5acac79dc02)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:10:47.643783 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:10:47.691198 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31480d73f44447679761a5acac79dc02', 'request_id': 'req-73a1df27-7833-46a9-989a-4244f972fa48'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.698724 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31480d73f44447679761a5acac79dc02', 'request_id': 'req-73a1df27-7833-46a9-989a-4244f972fa48'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.707503 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31480d73f44447679761a5acac79dc02', 'request_id': 'req-73a1df27-7833-46a9-989a-4244f972fa48'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.718553 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] The token cache is being invalidate because project 31480d73f44447679761a5acac79dc02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:10:47.718873 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.719508 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.721059 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1447/2921] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:10:47 2022] DELETE /identity/v3/projects/31480d73f44447679761a5acac79dc02 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:10:47.725592 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:10:47.727044 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:10:47.740568 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b65619-2dd3-4473-af15-f92136e2a688 admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:10:47.795796 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:10:47.796662 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:10:47.796948 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:10:47.797478 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] PATH_INFO: `/v3/users/ac3a4820a4b442e0b3b3f6cc36053cf2` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:10:47.798659 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3a4820a4b442e0b3b3f6cc36053cf2)` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:10:47.800621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:10:47.852501 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3a4820a4b442e0b3b3f6cc36053cf2'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.859492 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3a4820a4b442e0b3b3f6cc36053cf2'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.866987 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3a4820a4b442e0b3b3f6cc36053cf2'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.867621 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3a4820a4b442e0b3b3f6cc36053cf2'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.875340 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:10:47.881291 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.notifications [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3a4820a4b442e0b3b3f6cc36053cf2'} {{(pid=86640) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:47.888078 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1475/2922] 10.222.0.62 () {58 vars in 1223 bytes} [Wed Jun 15 16:10:47 2022] DELETE /identity/v3/users/ac3a4820a4b442e0b3b3f6cc36053cf2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:10:48.063388 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] Authenticating user token {{(pid=86639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:10:48.064864 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.fernet_utils [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:10:48.078098 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1df27-7833-46a9-989a-4244f972fa48 admin admin] Validating token access rules against request {{(pid=86639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:10:48.101341 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:10:48.138430 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38729f2fb8864984b35336009a142811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87109ec03f124710b1afe403b1e173c0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:10:48.139364 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:10:48.139650 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] SCRIPT_NAME: `/identity` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:10:48.139929 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] PATH_INFO: `/v3/projects/2cab9b0de3b445559fa85e7a5a2cd03b` {{(pid=86639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:10:48.141101 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cab9b0de3b445559fa85e7a5a2cd03b)` {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:10:48.143029 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] RBAC: Authorization granted {{(pid=86639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:10:48.189033 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cab9b0de3b445559fa85e7a5a2cd03b', 'request_id': 'req-d263fbcd-46d0-4c87-824f-5a74a25a747c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:48.195063 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cab9b0de3b445559fa85e7a5a2cd03b', 'request_id': 'req-d263fbcd-46d0-4c87-824f-5a74a25a747c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:48.202147 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cab9b0de3b445559fa85e7a5a2cd03b', 'request_id': 'req-d263fbcd-46d0-4c87-824f-5a74a25a747c'} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:48.213431 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] The token cache is being invalidate because project 2cab9b0de3b445559fa85e7a5a2cd03b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 16:10:48.213750 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:48.214378 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG keystone.notifications [None req-d263fbcd-46d0-4c87-824f-5a74a25a747c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 16:10:48.215953 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: [pid: 86639|app: 0|req: 1448/2923] 10.222.0.62 () {58 vars in 1232 bytes} [Wed Jun 15 16:10:48 2022] DELETE /identity/v3/projects/2cab9b0de3b445559fa85e7a5a2cd03b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 16:10:57.884306 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:10:58.210355 dv-cinder-845990-1-iscsi devstack@keystone.service[86639]: DEBUG dbcounter [-] [86639] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 16:14:17.011606 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] Authenticating user token {{(pid=86640) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 16:14:17.013071 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:14:17.159482 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c237ac5-d38d-470f-a790-ae80c00a316f admin admin] Validating token access rules against request {{(pid=86640) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 16:14:17.216026 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ea63b-d432-474e-bd32-80ce1b3ef9ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a786a95c48c348dbba269e0e33265271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aa8cd5804554314b5b0a61cb3c8ec9d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86640) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 16:14:17.216672 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ea63b-d432-474e-bd32-80ce1b3ef9ce service placement] REQUEST_METHOD: `GET` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 16:14:17.216952 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ea63b-d432-474e-bd32-80ce1b3ef9ce service placement] SCRIPT_NAME: `/identity` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 16:14:17.217226 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ea63b-d432-474e-bd32-80ce1b3ef9ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86640) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 16:14:17.219037 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.fernet_utils [None req-3d0ea63b-d432-474e-bd32-80ce1b3ef9ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86640) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 16:14:17.230720 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ea63b-d432-474e-bd32-80ce1b3ef9ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 16:14:17.232314 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ea63b-d432-474e-bd32-80ce1b3ef9ce service placement] RBAC: Authorization granted {{(pid=86640) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 16:14:17.242194 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: [pid: 86640|app: 0|req: 1476/2924] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Jun 15 16:14:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 16:14:27.237880 dv-cinder-845990-1-iscsi devstack@keystone.service[86640]: DEBUG dbcounter [-] [86640] Writing DB stats keystone:SELECT=89 {{(pid=86640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}